RL-TR-94-235 
Final  Technical  Report 
December  1994 


DECISION-THEORY  FOR 
CRISIS  MANAGEMENT 


Rockwell  International  Science  Center 


Sponsored  by 

Advanced  Research  Projects  Agency 
ARPA  Order  No.  7741 


OTIC 

elect E 
M^R  22  1995 


"'I 


APPROVED  FOR  PUBUC  RELEASE  DJSTR/BUT/ON  UNUM/TED. 


The  views  and  conclusions  contained  in  this  document  are  those  of  the  authors  and  should 
not  be  interpreted  as  necessarily  representing  the  official  policies,  either  expressed  or 
implied,  of  the  Advanced  Research  Projects  Agency  or  the  U.S.  Government. 


19950321  159 

Rome  Laboratory 
Air  Force  Materiel  Command 
Griff iss  Air  Force  Base,  New  York  , 


This  report  has  been  reviewed  by  the  Rome  Laboratory  Public  Affairs  Office 
(PA)  and  is  releasable  to  the  National  Technical  Information  Service  (NTIS) .  At 
NTIS  it  will  be  releasable  to  the  general  public,  including  foreign  nations. 

RL-TR-94-234  has  been  reviewed  and  is  approved  for  publication. 


APPROVED : 


ALBERT  G.  FRANTZ 
Project  Engineer 


FOR  THE  COMMANDER: 


HENRY  J.  BUSH 
Acting  Deputy  Director 

Command,  Control  &  Communications  Directorate 


If  your  address  has  changed  or  if  you  wish  to  be  removed  from  the  Rome  Laboratory 
mailing  list,  or  if  the  addressee  is  no  longer  employed  by  your  organization, 
please  notify  RL  (  C3CA  )  Griff iss  AFB  NY  13441.  This  will  assist  us  in  maintaining 
a  current  mailing  list. 

Do  not  return  copies  of  this  report  unless  contractual  obligations  or  notices  on  a 
specific  document  require  that  it  be  returned. 


Accesion  For 

NTIS 

CRA&I 

A 

DTIC 

TAB 

□ 

Unannounced 

□ 

Justification 

Bv  _ 

DECISION-THEORY  FOR  CRISIS  MANAGEMENT 

Distribution/ 

Moises  Goldszmidt 

Availability  Codes 

Adnan  Darwiche 

Avail  and  /  or 

Tom  Chavez 

Dist 

Special 

David  Smith 

James  White 

Contractor:  Rockwell  International  Science  Center 
Contract  Number:  F30602-91-C-  0031 
Effective  Date  of  Contract:  1  February  1991 
Contract  Expiration  Date:  30  September  1994 

Short  Title  of  Work:  Decision-Theory  for  Crisis 

Management 

Period  of  Work  Covered;  Feb  91  -  Sep  94 


Principal  Investigator: 

Phone : 

RL  Project  Engineer; 

Phone ; 


Moises  Goldszmidt 
(415)  325-1878 
Albert  G.  Frantz 
(315)  330-4031 


Approved  for  public  release;  distribution  unlimited. 


This  research  was  supported  by  the  Advanced  Research 
Projects  Agency  of  the  Department  of  Defense  and  was 
monitored  by  Albert  G.  Frantz,  RL  (C3CA) ,  525  Brooks  Rd, 
Griff iss  AFB  NY  13441-4505. 


REPORT  DOCUMENTATION  PAGE 


orm  Approved 
0MB  No.  0704-0188 


PubSc  reporting  burden  for  tHs  colection  of  rformation  is  estrristed  to  average  1  hour  per  resporTse,  rxLxfng  the  time  for  reviewing  n^udions,  searching  ®Psti^  dsta  source^ 

g^ing  and  rTiart»*ig  the  data  needed,  arxJ  corrpleting  arxJ  reviewing  the  colection  of  rforrriation.  Send  cortTnerts  regardng  the  burden  estimate  or  any  ether  asped  of  this 

colection  of  rformation,  hdudftg  suggestions  for  reduchg  this  btrden  to  WasHn^on  Headquarters  Services.  Directorate  for  irform^  Operations  andReports,  1 21 5  Jefferson 
Davis  Highway.  Sute  1 204,  Arln^on  VA  22202-4302,  and  to  the  Office  of  Managemert  and  Budg^  Paperwork  ReeUdon  Project  (0704^11 88),  WasHn^on  DC  20503. _ 

1 .  AGENCY  USE  ONLY  (Leave  Blank)  \z  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

Tinr-nmTnnr  1 QQA  Final  Feb  91  -  Sep  94 


2.  REPORT  DATE 
December  1994 


4.  TITLE  AND  SUBTITLE 


DECISION- THEORY  FOR  CRISIS  MANAGEMENT 


6.  AUTHOR(S)  ,  ,  ^  u  m  nu 

Moises  Goldszmidt,  Adnan  Darwiche,  Tom  Chavez, 

David  Smith,  and  James  White 


7.  PERFORMING  ORGANIZADON  NAME(S)  AND  ADDRESS(ES) 
Rockwell  International  Science  Center 
444  High  Street 
Palo  Alto  CA  94301 


5.  FUNDING  NUMBERS 
C  -  F30602-91-C-0031 
PE  -  62301E 
PR  -  G741 
TA  -  00 
WU  -  09 


a  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Advanced  Research  Projects  Agency 

3701  North  Fairfax  Drive  Rome  Laboratory  (C3CA) 

Arlington  VA  22203-1714  525  Brooks  Rd 

Griffiss  AFB  NY  13441-4505 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 


RL-TR-94-235 


1 1 .  SUPPLEMENTARY  NOTES 

Rome  Laboratory  Project  Engineer:  Albert  G.  Frantz/C3CA/ (315)  330-4031 

12a  DISTRIBUHON/AVAILABIUTY  STATEMENT 

12b.  DISTRIBUHON  CODE 

Approved  for  public  release;  distribution  unlimited. 

1  3.  ABSTRACT  200  words) 

The  Rockwell  International  Science  Center's  main  contribution  to  the  ARPA/Rome 
Laboratory  Planning  Initiative  has  been  the  development  and  application  of  decision 
theoretic  concepts  and  tools  for  the  problem  of  military  transportation  and  crisis 
planning.  Real  world  planning,  like  most  complex  problems,  involves  multiple  com¬ 
peting  objectives  and  uncertainty  about  actual  outcomes.  Thus,  effective  operational 
tools  for  planning  must  be  able  to  efficiently  manage  uncertainty  and  trade-offs 
between  competing  objectives.  Rockwell's  contributions  span' three  main  areas  of  the 
planning  process:  plan  generation,  plan  evaluation,  and  plan  simulation.  Contribu¬ 
tions  range  from  basic  research  results  to  application-oriented  software  tools  for 
military  transportation  planning. 


14.  SUBJECT  TERMS 


Planning,  Decision  theory.  Uncertainty 


1  a  NUMBER  OF  PAGES 

76 _ 

16.  PRICE  CODE 


17.  SECURITY  CLASSIFICATION 
OF  REPORT 

UNCLASSIFIED 

NSN  754(KI1 -280-5500 


1  a  SECURITY  CLASSIFICATION  1 9.  SECURITY  CLASSIFICATION  20.  UMITAHON  OF  ABSTRACT 
OF  THIS  PAGE  OF  ABSTRACT 

UNCLASSIFIED  UNCLASSIFIED  UL 

standard  Form  298  (Rev.  2-89) 
Preserbed  by  ANSI  Std  Z39-1 8 
298-102 


Contents 

1  Executive  Summary  1 

1.1  Plan  Generation  Results .  1 

1.2  Plan  Evaluation  Results  . .  2 

1.3  Plan  Simulation  Results .  3 

2  Introduction:  Decision-Theory  for  Crisis  Management  4 

3  Plan  Generation  ^ 

3.1  Conditional  Planning .  6 

3.2  Control  of  Planning  Search .  ^ 

3.2.1  Operator  graphs  .  8 

3.2.2  Operator  relevance .  10 

3.2.3  Variable  analysis .  10 

3.2.4  Open  condition  ordering . 12 

3.2.5  Recursion  analysis  .  13 

3.2.6  Postponing  Conflicts .  14 

3.2.7  Conflict  resolution  strategies .  15 

3.3  Partial  Plan  Evaluation .  10 

4  Plan  Evaluation  18 

4.1  Transportation  Trade-off  Analyzer .  20 

4.2  Decision-Theoretic  Modeling  Tools  in  the  CPE .  20 

4.3  Course  Of  Action  Trade-off  Analyzer . 22 

4.4  Sensitivity  Analysis  in  Planning .  25 

4.4.1  The  Expected  Value  of  Information . 26 

4.4.2  Computing  the  Value  of  Information  in  Large  Decision  Models  .  .  26 

5  Plan  Simulation  29 

5.1  Action  Networks  .  30 

5.1.1  Introduction  into  action  networks .  31 

5.1.2  Progress  on  Action  Networks  .  31 

5.2  Algorithms .  33 

5.2.1  Introduction . 33 

5.2.2  Dynamic  Conditioning .  35 

5.2.3  Prediction  Algorithm:  Specific  to  action  networks .  36 

5.2.4  Prediction  Algorithm:  Specific  to  order-of-magnitude  networks  .  .  37 

5.2.5  Bounded  Conditioning .  37 

i/li 


39 


6  Concluding  Remarks 


List  of  Figures 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 


13 

14 

15 

16 
17 


A  partial  conditional  plan .  7 

Operator  graph  for  a  simple  propositional  problem .  9 

First-order  Operator  graph .  11 

Operator  graph  after  variable  analysis .  12 

Partial  plans  with  recursion .  14 

Search  space  relationships  for  five  threat  removal  strategies.  .  16 

Propositional  operator  graph  with  operator  costs .  17 

Partial  plan  with  two  open  conditions .  18 

Model  structure  used  in  the  Transportation  Trade-Off  Analyzer .  19 

An  example  of  an  influence  diagram  in  IDEAL-EDIT .  21 

The  graph  describes  the  buildup  of  U.S.  citizens  at  an  assembly  area.  .  .  22 

Top  level  model  for  the  NEO  course  of  action  analysis  tool.  The  tool 
generates  graphs  describing  riSks  for  U.S.  citizens  and  estimates  of  trans¬ 
portation  times .  23 

Conceptual  overview  of  the  algorithm  for  estimating  EVI .  27 

Probability  distribution  on  Z .  28 

Schematic  view  of  a  plan  simulation  scenario .  30 

An  example  of  an  action  network .  32 

Current  realization  of  the  Plan  Simulator  and  Analyzer  (PSA)  tool.  ...  35 


List  of  Tables 

1  Summary  of  Rockwell’s  contributions  to  ARPI. .  1 

2  Impact  on  scientific  and  applications  communities  and  future  plans.  ...  2 

3  Operator  definitions .  8 


iv 


1  Executive  Summary 


The  Rockwell  Science  Center’s  main  contribution  to  the  ARPA  Planning  Initiative 
(ARPI)  has  been  the  development  and  application  of  decision  theoretic  concepts  and 
tools  to  the  problem  of  military  transportation  and  crisis  planning.  Real  world  planning, 
like  most  complex  problems,  involves  multiple  competing  objectives  and  uncertainty 
about  actual  outcomes.  Thus,  effective  operational  tools  for  planning  must  be  able  to 
efficiently  manage  uncertainty  and  trade-offs  between  competing  objectives.  Rockwell’s 
contributions  span  three  main  areas  of  the  planning  process:  generation,  evaluation, 
and  simulation.  Contributions  range  from  basic  research  results  to  application  oriented 
software  tools  for  military  transportation  planning.  These  contributions  are  briefly  enu¬ 
merated  in  the  sections  below,  and  are  summarized  in  Tables  1  and  2. 


Planning 

Subtask 

Basic  Advances 

Demonstrations  &  Deliverables 

Generation 

Conditional  planning  (First  CNLP). 
Control  of  planning  search  (Op.  Graphs). 
Evaluation  of  partial  plans. 

Briefing  at 

Rome  Labs  1993. 

Evaluation 

Transportation  Trade-off 

Analyzer  Tool  (TTA). 

Course  of  Action  Trade-off 

Analyzer  Tool  (COATA). 

New  Algorithms  for 

Sensitivity  Analysis. 

lFD-3  Sept  1993. 

ARPI  Workshops 

1992,  1993,  and  1994. 
Software  training  sessions 
for  Rome  Labs  Manager, 
Sept.  1993 

TTA,  IDEAL, 

IDEAL-DEMOS  May  1992. 

Simulation 

Uncertainty  modeling  over  time. 

Suite  of  approximate  and  exact 
algorithms  for  inference. 

Site  visit  (Palo  Alto)  by 

Rome  Labs  Manager, 

Sept  1994. 

Table  1:  Summary  of  Rockwell’s  contributions  to  ARPI. 


1.1  Plan  Generation  Results 

•  Conditional  planning.  Developed  and  implemented  the  first  algorithm  for  condi¬ 
tional  partial  order  planning.  The  algorithm  accepts  a  goal,  initial  conditions,  and 
Strips-like  operators,  extended  to  allow  uncertain  outcomes.  It  produces  partially- 
ordered  plans  with  conditional  branches  for  observed  uncertainties.  Details  of  the 
algorithm  are  in  [37].  The  ideas  from  this  algorithm  are  now  used  in  several  other 
planners  and  algorithms  including  [18,  20,  8].  See  Section  3.1. 
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Planning 

Subtask 

Impact 

Future 

Plans 

Generation 

Techniques  and  methods  influenced 
other  planners  such  as 
[18,  20,  8,  27,  34,  29]. 

Techniques  being  extended  in 
ARPA-MADE  program. 

Part  of  Rockwell  IR&D. 

Evaluation 

Decision  theoretical 

component  in  TARGET. 

Graphical  interfaces  being 

extended  for  in-house  tools. 

Simulation 

Uncertainty  modeling  techniques 
being  used  by  other 
researchers  [25,  1]. 

Extend  simulation  techniques  to 
generation. 

Applications  to  diagnosis 

for  Rockwell  business  divisions. 

Table  2:  Impact  on  scientific  and  applications  communities  and  future  plans. 

•  Control  of  planning  search.  Developed,  implemented,  and  evaluated  a  number 
of  techniques  to  help  control  the  combinatorial  explosion  that  occurs  m  genera¬ 
tive  planning.  The  control  techniques  include  smart  ordering  of  open  conditions, 
recognition  and  control  of  recursion,  variable  binding  analysis,  and  smart  resolu¬ 
tion  of  threats  between  operators.  Descriptions  of  these  techniques  can  be  found 
in  [42,  44,  38,  41].  Several  of  these  ideas  are  now  being  investigated  in  other 
generative  planners  [27,  34,  29].  See  Section  3.2. 

•  Evaluation  of  partial  plans.  Developed  a  technique  for  estimating  the  value 
of  partially  completed  plans.  This  technique  provides  estimates  of  the  cost  and 
probability  of  success  of  achieving  different  possible  subgoals  that  could  occur  for 
a  problem.  This  information  is  then  used  to  compute  an  estimate  of  the  cost 
and  probability  of  success  for  a  partial  plan  [41].  We  are  currently  investigating 
this  technique  further  for  the  generation  of  assembly  plans  in  the  ARPA  MADE 
project.  See  Section  3.3. 

1.2  Plan  Evaluation  Results 

•  Software  tools.  During  the  course  of  the  program  Rockwell  produced  and  made 
available  to  the  ARPI  community  a  set  of  tools  for  utility  modeling  and  decision 
theoretic  evaluation  of  plans.  These  include: 

-  IDEAL,  IDEAL-DEMOS.  Domain  independent  decision  theoretic  soft¬ 
ware  tools,  which  enable  construction  and  evaluation  of  utility  models  (made 
available  to  CPE  in  May  1992).  Delivered  to  the  Common  Prototyping  En¬ 
vironment  (CPE)  (1992).  See  Section  4.2. 

—  Transportation  Trade-off  Analyzer  (TTA).  Utility  models  and  inference 
machinery  for  computing  trade-offs  between  consumption  of  resources  and 
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the  expected  time  for  the  achievements  of  goals  under  uncertainty  in  the 
transportation  domain  (delivered  in  May  1992).  See  Section  4.1. 

-  Course  Of  Action  Trade-ofF  Analyzer  (COATA).  Utility  models  and  in¬ 
ference  machinery  for  the  evaluation  of  courses  of  action  in  the  noncombatant 
evacuation  domain.  COATA  constituted  the  decision  theoretic  component  of 
the  “Theater-level  Analysis,  Replanning  and  Graphical  Execution  Toolbox” 
(TARGET)  for  the  third  Integrated  Feasibility  Demonstration  (IFD-3)  (May 
and  September  1993).  See  Section  4.3. 

•  Sensitivity  analysis.  Developed  a  new  approximation  algorithm  for  uncovering 
relevant  data  with  respect  to  a  given  set  of  decisions  and  a  course  of  actions.  This 
functionality  has  proven  to  be  essential  for  building  utility  models  in  both  COATA 
and  the  TTA.  See  Section  4.4. 


1.3  Plan  Simulation  Results 

•  Modeling  uncertain  domains.  Developed  a  graphical  language,  called  Action 
Networks,  for  modeling  uncertain  situations  for  the  purpose  of  simulating  their 
dynamics  under  diffei'ent  courses  of  action.  Action  networks  are  based  on  extending 
causal  networks  with  explicit  constructs  for  time  and  action.  See  Section  5.1. 

•  Algorithms  for  uncertain  inference.  Developed  a  collection  of  algorithms  for 
simulating  plans  under  uncertain  conditions.  The  algorithms  allow  the  user  to 
trade-off  accuracy  for  response  time  in  the  simulation  process,  and  they  can  be 
used  in  general  forecasting  of  uncertainty  models: 

—  e-bounded  conditioning.  A  method  for  the  approximate  simulation  of 
plans  that  allows  the  user  to  trade  the  exactness  of  the  simulation  with  the 
time  it  takes  to  compute  the  simulation.  See  Section  5.2.5. 

—  Dynamic  Conditioning.  A  method  for  the  exact  simulation  of  plans  based 
on  hypothetical  reasoning  (reasoning  by  cases).  Dynamic  conditioning  pro¬ 
vides  an  effective  infra-structure  for  realizing  computational  techniques  that 
are  based  on  reasoning  by  cases,  e-bounded  conditioning  makes  use  of  dy¬ 
namic  conditioning.  See  Section  5.2.2. 

—  Kappa  Predict.  An  algorithm  for  predicting  the  effect  of  plans  with  re¬ 
spect  to  order-of-magnitude  action  networks.  The  algorithm  has  a  polynomial 
computational  complexity  but  may  be  unable  to  provide  complete  answers  in 
certain  cases.  Although  the  answers  it  provides  may  be  incomplete,  they  may 
be  enough  to  judge  the  quality  of  certain  plans.  It  is  a  major  building  block 
of  e-bounded  conditioning.  Section  5.2.4. 

—  Action  Network  Predict.  An  algorithm  for  predicting  the  effect  of  plans 
with  respect  to  a  subclass  of  action  networks.  The  algorithm  has  a  polynomial 
computational  complexity  on  this  subclass.  See  Section  5.2.3. 
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2  Introduction:  Decision-Theory  for  Crisis  Man¬ 
agement 

Military  planning  for  crisis  management  and  transportation  scheduling  is  characterized 
by  unavoidable  uncertainties  and  difficult  trade-offs  about  resources  and  objectives.  For 
example,  reliability  and  rapidity  of  deployment  often  come  at  the  cost  of  increased 
resource  requirements,  including  transport  crew,  vehicles,  fuel,  port  facilities,  and  so  on. 
Under  limited  resources  and  in  crisis  conditions,  prompt  delivery  of  force  A  to  location 
X  may  require  delays  in  delivery  for  unit  B  to  location  Y.  Greater  transport  volumes 
may  be  possible  for  given  resources,  but  only  at  the  cost  of  higher  risks  of  delay  or 
loss.  Thus,  any  successful  tool  for  planning  in  this  domain  must  be  able  to  represent 
the  inherent  uncertainty,  and  formulate,  weight  and  resolve  the  different  tiade-offs  in 
order  to  compute  and  evaluate  feasible  courses  of  actions.  Rockwell  s  main  objective  in 
this  program  has  been  to  develop  computational  models  and  systems  that  incorporate 
notions  of  rational  decision  making  and  reasoning  under  uncertainty  in  order  to  enable 
the  generation,  evaluation  and  simulation  of  military  plans  that  effectively  meet  these 
demands. 

An  examination  of  the  state  of  the  art  at  the  time  this  program  began  reveals  that 
traditional  approaches  to  planning  and  scheduling,  including  the  current  militaiy  sys¬ 
tems,  employ  goal-oriented  and  constraint  satisfaction  notions  that  deal  primarily  with 
feasibility  and  goal  satisfaction.  These  approaches  include  a  specification  of  a  set  of 
Boolean  goals,  a  deterministic  world  model,  and  a  set  of  action  descriptions  specifying 
how  an  action  deterministically  transforms  the  world  from  one  state  to  another.  Tra¬ 
ditional  AI  planning  shares  this  world  view  —  it  is  normally  defined  as  generating  a 
sequence  or  program  of  actions  that  will  allow  the  agent  to  transform  the  current  world 
into  a  world  where  the  goal  is  true.  This  view  of  planning  is  too  narrow  as  a  general 
notion  of  planning  for  several  reasons. 

The  first  problem  is  that  goals  are  rarely  expressible  as  a  set  of  Boolean  sentences. 
Invariably,  there  are  partially  achievable  goals  and  trade-offs  involved.  We  wish  to 
achieve  security  for  American  citizens  in  some  region,  but  security  is  rarely  absolute 
and  it  may  only  be  possible  to  secure  major  population  centers.  Given  these  types 
of  trade-offs,  we  believe  it  is  necessary  to  employ  a  richer  representation  of  objectives, 
namely,  a  utility  function,  that  allows  one  to  score  various  combinations  of  outcomes 
(e.g.,  casualties,  fiscal  expenditures,  achievement  of  mission  objectives).  As  soon  as 
one  adopts  a  utility  or  objective  function  (or  even  some  partial  order  on  world  states) 
as  a  representation  of  preferences,  planning  is  transformed  from  a  notion  of  constraint 
satisfaction  and  goal  achievement  into  an  optimization  problem. 

The  second  problem  is  that  most  automated  planning  and  scheduling  systems,  in¬ 
cluding  techniques  arising  in  both  AI  and  operations  research  (e.g.  linear  programming), 
do  not  include  any  notion  of  uncertainty.  AI  planning  systems  typically  assume  a  deter¬ 
ministic  world  model,  where  all  actions  have  deterministic  effects,  and  any  changes  to  the 
world  model  are  due  to  the  actions  of  the  agent.  These  assumptions  are  clearly  untenable 
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for  military  planning  and  scheduling.  Action  effects  are  not  guaranteed  because  planes 
can  break  down,  ports  can  be  bombed,  and  weather  can  be  adverse,  all  in  unpredictable 
ways.  Other  agents,  such  as  the  enemy  or  some  other  threat,  can  interfere  with  ongoing 
plans  or  actions  in  a  non-deterministic  manner.  Therefore,  mechanisms  for  addressing 
uncertainty  are  necessary  in  a  planning  situation.  To  deal  with  this  problem,  existing 
DOD  planning  exercises  often  assume  a  worst-case  scenario  in  order  to  assess  the  impact 
of  an  adverse  outcome  on  the  plan.  Unfortunately,  this  strategy  yields  inflexible  plans 
with  sometimes  unacceptable  economical  costs.  In  this  project,  we  adopt  probability  as 
a  mechanism  for  modeling  uncertainty.  This  provides  a  theoretically  sound  means  for  in¬ 
corporating  uncertainty,  and  allows  us  to  utilize  a  number  of  well-established  algorithms 
for  reasoning. 

Rockwell  activities  in  this  program  cover  three  main  areas  in  planning  technology: 

1.  Plan  Generation.  The  objective  behind  this  area  of  planning  has  been  to  re¬ 
search  and  develop  general  purpose  planning  systems  suitable  for  military  trans¬ 
portation  planning  and  crisis  management  based  on  generalizing  classical  gener¬ 
ative  planning  techniques  to  incorporate  utility  and  probability  metrics.  Several 
significant  advances  were  made  that  led  to  scholarly  papers.  Upon  publication 
these  advances  were  adopted  by  other  researchers  in  their  automated  generative 
planners. 

2.  Plan  Evaluation.  The  work  in  this  area  concentrated  on  developing  approaches 
and  software  tools  for  the  construction,  analysis,  and  explanation  of  utility  models 
in  the  domain  of  military  transportation  planning  and  crisis  management.  Some 
of  these  models  were  developed  in  close  collaboration  with  military  personnel. 

3.  Plan  Simulation.  Our  involvement  on  plan  simulation  has  been  directed  towards 
creating  a  tool  for  supporting  and  assisting  a  military  planner  in  simulating  the 
behavior  of  each  possible  course  of  action  to  enable  a  choice  of  which  one  to 
adopt.  This  work,  under  the  name  Action  Networks,  has  significantly  extended 
the  power  of  the  probabilistic  reasoning  methods  with  the  addition  of  capabilities 
for  qualitative  reasoning,  the  explicit  representation  of  time  and  actions,  and  the 
increased  ease  of  end-user  domain  modeling. 

A  summary  of  the  technical  approach  and  main  results  in  each  of  these  areas  can  be 
found  in  Sections  3,  4,  and  5  respectively.  The  details  can  be  found  in  the  set  of  papers 
we  include  in  the  Appendix. 


3  Plan  Generation 

Our  work  on  plan  generation  has  focused  on  three  areas: 

1.  Extension  of  generative  planning  techniques  to  allow  the  generation  of  conditional 
plans  and  plans  under  uncertainty. 
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2.  Controlling  the  search  required  to  generate  plans. 

3.  Using  utility  information  to  help  guide  a  planner  towards  better  plans. 

We  have  made  significant  progress  in  all  three  of  these  areas  and  give  brief  descrip¬ 
tions  below. 


3.1  Conditional  Planning 

Classical  goal  based  planners  assume  that  1)  the  effects  of  operators  are  determinis¬ 
tic  and  completely  specified,  and  2)  no  exogenous  events  or  actions  take  place  during 
plan  execution.  Because  of  these  assumptions,  classical  planners  generate  unconditional 
deterministic  plans. 

A  conditional  plan  is  necessary  when  uncertainties  in  the  environment  or  in  the 
effects  of  actions  preclude  the  selection  of  a  single  course  of  action  to  accomplish  a  goal. 
A  conditional  plan  contains  tests  and  branches  that  result  in  different  courses  of  action 
depending  on  the  outcome  of  each  test. 

Under  this  contract  we  developed  the  first  algorithm  for  conditional  partial  order 
planning  (CNLP).  The  algorithm  is  based  on  the  Systematic  Non-Linear  Planning  al¬ 
gorithm  (SNLP)  of  McAllester  and  Rosenblitt  [31].  Like  SNLP,  CNLP  accepts  a  goal, 
initial  conditions,  and  Strips-like  operators.  However,  unlike,  SNLP,  operators  can  have 
several  different  sets  of  effects.  Each  set  of  effects  denotes  a  different  possible  outcome 
for  the  operator.  For  example  the  operator: 

Drive(a:,?/) 

Pre:  {At(a;),  Route(a:,y)} 

Effects:  {At(y),  ->At(a:)} 

{Stuck(a:,y),  -'At(a:)} 

has  two  possible  outcomes,  one  where  the  destination  is  attained,  and  one  where  the 
vehicle  gets  stuck  enroute.  (One  can  imagine  other  possible  outcomes.) 

Like  the  SNLP  algorithm,  CNLP  works  backwards  from  open  conditions  (goals  and 
preconditions)  and  inserts  new  steps  and  causal  links  into  the  plan  to  achieve  those  open 
conditions.  For  CNLP,  an  action  can  be  inserted  into  the  plan  if  any  one  of  its  effect  sets 
would  achieve  the  open  condition.  When  a  step  is  introduced  with  multiple  outcomes, 
CNLP  needs  to  consider  how  to  achieve  the  goal  if  any  of  the  alternative  (contingent) 
outcomes  were  to  occur. 

To  do  this,  CNLP  must  keep  track  of  two  additional  kinds  of  information: 

1.  Context:  the  set  of  contingencies  under  which  each  action  in  the  plan  will  be 
executed. 

2.  Reason:  the  ultimate  purpose  for  each  action  and  open  condition  in  the  plan. 
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Context  82:? 


Figure  1:  A  partial  conditional  plan. 

When  a  step  with  multiple  outcomes  is  inti’oduced,  each  outcome  is  tagged  with  a 
different  context  label.  This  context  indicates  the  conditions  under  which  any  dependent 
steps  will  be  executed,  so  any  action  supported  by  that  effect  inherits  the  context  label. 

To  plan  for  contingent  outcomes,  the  planner  reintroduces  the  original  goal  as  an 
open  condition,  but  gives  it  a  reason  label  that  corresponds  to  the  contingent  context 
being  considered.  Planning  then  proceeds  as  before,  except  that  steps  in  the  original 
context  cannot  be  used  to  achieve  open  conditions  with  the  new  reason  label. 

As  an  example,  consider  the  partial  plan  in  Figure  1.  The  original  goal  is  the  single 
clause  G,  and  the  step  is  being  used  to  achieve  it.  The  step  S2  is  being  used  to  achieve 
the  precondition  P  of  ^i.  However,  ^2  has  another  possible  outcome,  Q.  The  step  Si  is 
therefore  labelled  with  the  context  S2  :  P.  CNLP  also  adds  a  new  open  condition  G' 
(a  copy  of  the  goal  G)  with  reason  S2  '■  Q-  Because  S2  '■  P  and  S2  ■  Q  are  incompatible, 
the  step  Si  cannot  be  used  to  achieve  G'. 

Threat  resolution  for  CNLP  is  also  more  complex  than  for  SNTP.  For  example, 
threats  cannot  occur  between  incompatible  contexts.  Conditioning  an  action  on  a  con¬ 
tingent  outcome  can  therefore  be  used  to  remove  a  threat  from  a  plan. 

Detailed  descriptions  of  CNLP  can  be  found  in  [37]  and  [36].  Techniques  developed 
for  CNLP  are  now  being  used  in  several  other  planners  including  [18],  [20],  and  [8]. 


3.2  Control  of  Planning  Search 

Control  of  search  is  a  critical  problem  in  generative  planning.  The  trouble  is  that  there 
are  too  many  partial  plans,  most  of  which  either  fail,  or  are  very  costly.  Unfortunately, 
the  introduction  of  uncertainty  and  conditionals  only  makes  the  problem  worse;  there 
are  more  circumstances  that  can  occur  and  therefore  more  possibilities  that  must  be 
considered. 
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We  have  developed  and  investigated  a  number  of  techniques  to  help  control  search 
including: 

1.  Recognition  of  operator  relevance, 

2.  Variable  binding  analysis, 

3.  Smart  open  condition  ordering, 

4.  Recursion  analysis, 

5.  Conflict  postponement, 

6.  Smart  conflict  resolution  strategies. 

Most  of  these  techniques  make  use  of  a  structure  called  an  operator  graph  (also 
developed  under  this  contract).  Operator  graphs  facilitate  meta-level  reasoning  about 
the  way  in  which  relevant  operators  can  interact.  In  the  next  section  we  briefly  describe 
operator  graphs.  In  subsequent  sections  we  briefly  describe  each  of  the  above  control 
techniques.  Several  of  these  control  techniques  are  now  being  investigated  by  other 
researchers  including  Draper,  Joslin  [27],  Kambhampati  and  Weld. 

3.2.1  Operator  graphs 

An  operator  graph  is  a  directed  graph  consisting  of  operators  and  their  preconditions. 
Edges  in  the  graph  indicate  which  preconditions  belong  to  which  operators,  and  which 
operators  can  potentially  be  used  to  satisfy  which  preconditions.  To  illustrate,  consider 
the  simple  set  of  operators  in  the  table  below: 


Operator 

Precondition 

Effects 

0i 

R 

A 

O2 

S 

A 

O3 

T 

B 

O4 

u,v 

B 

O5 

W 

T 

Table  3:  Operator  definitions. 

Suppose  that  the  goal  is  A  and  B  and  the  initial  conditions  are  72,  S,  t/ ,  V,  and 
W.  The  operator  graph  for  this  problem  is  shown  in  Figure  2.  Intuitively  the  operator 
graph  is  a  schematic  that  shows  the  basic  relationships  between  the  goals,  subgoals  and 
operators  for  this  problem.  Among  other  things,  it  shows  that  both  0\  and  O2  can 
potentially  be  used  to  achieve  A,  and  that  O3  and  O4  can  potentially  be  used  to  achieve 

B. 
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Figure  2:  Operator  graph  for  a  simple  propositional  problem. 
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While  there  may  be  many  potential  plans  for  a  problem  there  is  only  one  operatoi 
graph,  and  each  operator  appears  only  once  in  the  graph.  As  a  result  the  operator  graph 
has  bounded  size  and  is  quick  and  easy  to  construct.  A  more  complete  description  ot 
operator  graphs  and  their  construction  can  be  found  in  [44]. 

3.2.2  Operator  relevance 

One  important  advantage  of  constructing  an  operator  graph  for  a  problem  is  that  it 
allows  us  to  easily  determine  which  operators  are  relevant  to  each  open  condition  m  a 
planning  problem.  For  the  open  condition  B  we  can  readily  determine  that  O3  and  O4 
are  the  only  relevant  operators. 

We  can  take  additional  advantage  of  this  property  by  performing  some  analysis  on 
the  operator  graph  ahead  of  time.  Suppose  that  in  the  example  above,  W  were  not  in 
the  open  conditions.  In  this  case  there  would  be  no  way  of  achieving  the  precondition  of 
Os  so  Os  can  be  deleted  from  the  graph.  Having  done  this,  there  is  no  way  of  achieving 
r,’the  precondition  of  O3.  As  a  result,  O3  can  also  be  deleted  from  the  graph.  More 
generally,  if  any  precondition  of  an  operator  in  the  operator  graph  cannot  be  achieved, 
the  operator  can  be  deleted  from  the  graph.This  can  be  applied  repeatedly  to  eliminate 
all  irrelevant  operators  from  the  graph. 

Doing  this  analysis  reduces  the  number  of  blind  alleys  that  the  planner  must  inves¬ 
tigate. 

3.2.3  Variable  analysis 

The  example  used  above  was  purely  propositional,  but  many  planning  problems  involve 
variables.  Figure  3  shows  an  operator  graph  for  a  first-order  version  of  the  previous 
problem.  As  indicated  in  the  graph,  there  are  many  initial  conditions  that  satisfy  each 
of  the  predicates  R{x),  5(x),  W(x),  U{x,y)  and  V{y).  Now  suppose  that  our  planner 
first  chooses  to  work  on  the  open  condition  A(x),  and  chooses  the  plan  where  0i  is 
used  to  achieve  A{x).  The  planner  might  then  try  different  possible  bindings  for  x 
that  achieve  R{x),  and  then  try  to  achieve  B{x).  Unfortunately,  none  of  the  possible 
bindings  for  VF(x)  are  compatible  with  the  bindings  for  R{x).  Similarly,  none  of  the 
possible  bindings  for  U{x,y)  are  compatible  with  the  bindings  for  V{y).  As  a  result,  all 
of  these  plans  will  ultimately  fail. 

Rather  than  rediscovering  this  for  each  plan,  this  variable  binding  analysis  can  be 
performed  once  in  the  operator  graph.  We  start  at  the  bottom  of  the  graph  and  work 
upwards,  figuring  out  the  possible  bindings  for  each  open  condition  in  the  graph.  So,  in 
our  example  the  possible  bindings  for  A(x)  would  be  x  G  0, 2, 4,6,8,10...,  and  for  r(x) 
would  be  X  G  1,3,5,....  The  bindings  for  C/(x,y)  and  V{y)  are  incompatible,  so  O4 
can  be  immediately  removed  from  the  graph,  and  the  possible  bindings  for  B{x)  become 
a;  G  1,3,5,....  Many  of  the  bindings  for  A(x)  and  B{x)  are  incompatible.  Taking  the 
intersection,  we  get  the  set  x  G  5, 15, 25, . . .. 
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xe  {5,15,25,...} 


xe  {5,15,25,...} 


xe  {5,  15,25,...} 


Figure  4:  Operator  graph  after  variable  analysis. 

The  second  part  of  the  process  is  to  propagate  these  limitations  back  down  the  tree. 
When  we  do  this  for  A{x)  we  find  that  the  remaining  bindings  are  incompatible  with  the 
bindings  for  R{x),  so  Oi  can  be  deleted  from  the  graph.  When  the  process  is  complete, 
the  resulting  operator  graph  is  as  shown  in  Figure  4.  This  operator  graph  dramatically 
limits  the  space  of  plans  that  will  have  to  be  explored  by  the  planner. 

In  general,  variable  analysis  in  the  operator  graph  can  be  expensive  because  the  sets  of 
variable  bindings  can  become  large.  However  it  is  always  possible  to  simplify  the  binding 
sets  by  removing  those  variables  that  have  too  many  possible  solutions..  Removing  the 
variable  means  that  we  are  assuming  it  can  take  on  any  possible  value.  This  may  reduce 
the  amount  of  pruning  that  can  be  done,  but  doesn’t  affect  the  completeness  or  soundness 
of  the  planning  process. 

3.2.4  Op  en  condition  ordering 

The  choice  of  which  open  condition  to  work  on  next  often  results  in  orders  of  magnitude 
difference  in  the  amount  of  search  required.  The  most  important  factor  in  deciding  which 
open  condition  to  work  on  next  is  the  number  of  different  possible  ways  of  achieving  the 
open  condition.  If  there  are  few  potential  plans  for  an  open  condition,  then  choosing 
and  expanding  that  open  condition  does  not  significantly  increase  the  number  of  partial 
plans.  Alternatively,  expanding  an  open  condition  with  many  options  leads  to  many 


12 


possible  plans. 

Counting  the  number  of  possible  expansions  for  each  open  condition  does  add  over¬ 
head  to  the  planning  process.  However,  the  operator  graph  can  be  used  to  help  speed 
this  process.  First,  it  provides  an  upper  bound  on  the  number  of  possibilities  for  each 
open  condition.  Second,  it  allows  the  planner  to  rapidly  find  and  count  the  relevant 
expansions  for  each  open  condition.  As  a  result,  the  overhead  for  this  technique  seems 
to  be  minimal.  We  have  evaluated  this  technique  in  several  standard  planning  test  do¬ 
mains  and  found  it  to  be  extremely  successful.  These  experimental  results  can  be  found 
in  [38].  Joslin  [27]  has  done  further  evaluation  of  this  technique  and  has  confirmed  its 
significance. 

3.2.5  Recursion  analysis 

Many  operators  in  planning  problems  allow  recursion.  For  example,  a  transportation 
operator  might  be  used  to  go  from  A  to  R,  but  might  also  be  used  to  go  back  from  B 
to  A.  This  means  that  a  plan  can  be  constructed  that  involves  going  back  and  forth 
between  A  and  B  any  number  of  times.  Sometimes  this  may  be  necessary.  For  example, 
several  trips  might  be  necessary  to  transport  all  the  supplies  to  B.  Unfortunately,  such 
operators  also  allow  planners  to  construct  and  investigate  lots  of  ridiculous  plans. 

We’ve  developed  a  technique  that  helps  to  avoid  useless  recursion.  The  basic  idea  is 
to  recognize  when  the  planner  has  generated  a  repeated  subgoal  and  suspend  work  on 
that  subgoal  until  there  is  some  other  reason  for  exploring  that  repeated  subgoal.  To 
illustrate,  consider  the  partial  plan  in  Figure  5a.  The  goal  is  to  achieve  P  and  7?,  and 
the  step  is  used  to  achieve  P.  S2  is  used  to  achieve  the  precondition  Q  oi  Si,  but  S2 
has  the  precondition  P,  which  is  one  of  the  goals  the  planner  was  trying  to  achieve  in  the 
first  place.  As  a  result,  the  open  condition  P  of  S2  can  be  suspended.  This  means  that 
the  planner  should  not  work  on  this  open  condition  unless  some  other  open  condition  is 
linked  into  either  Si  or  S2-  In  Figure  5b,  the  open  condition  R  is  now  being  established 
by  S2-  As  a  result,  the  condition  P  must  be  reenabled.  In  contrast,  if  no  other  open 
condition  is  ever  linked  to  or  S2  (as  in  Figure  5c),  the  plan  is  discarded. 

There  is  considerable  overhead  involved  in  recognizing  recursive  open  conditions,  and 
in  checking  to  see  when  they  should  be  re-enabled.  Operator  graphs  help  considerably 
in  this  process  because  they  make  it  clear  which  open  conditions  are  susceptible  to 
recursion,  and  also  make  it  clear  which  operators  could  potentially  link  into  a  recursion 
and  cause  open  conditions  to  be  re-enabled.  (If  there  is  no  possibility  of  such  links,  the 
plan  can  be  discarded  early.) 

There  are  a  number  of  subtleties  involved  in  the  general  technique. 

1.  The  presence  of  variables  makes  suspension  and  enablement  conditions  more  com¬ 
plex. 

2.  Certain  conflicts  between  operators  may  also  require  re-enablement  of  suspended 
open  conditions. 
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Figure  5:  Partial  plans  with  recursion. 

A  more  complete  description  of  this  technique  can  be  found  in  [40]. 

3,2.6  Postponing  Conflicts 

During  the  planning  process,  conflicts  often  arise  between  operators  used  to  achieve 
different  parts  of  a  goal.  For  example,  one  operator  may  consume  a  resource  needed  for 
another  operator,  or  may  change  some  condition  in  the  world  that  is  a  prerequisite  for 
another  operator.  Resolving  these  conflicts  is  therefore  a  critical  part  of  the  planning 
process. 

Several  planners  [39],  [49],  [50]  ignore  conflicts  until  planning  is  otherwise  complete, 
and  then  try  to  fix  the  plan  to  eliminate  conflicts.  If  the  subgoals  in  the  problem  interact 
only  loosely,  this  can  be  an  efficient  strategy.  However,  if  there  are  complex  interactions 
between  the  subgoals  and  operators  for  a  problem,  this  approach  results  in  extensive 
backtracking  by  the  planner. 

An  alternative  approach  used  in  recent  planning  systems  [31],  [2],  [52],  [8]  is  to  resolve 
each  conflict  as  it  arises  during  the  planning  process.  Using  this  approach  a  planner 
notices  unresolvable  conflicts  early  in  the  planning  process.  The  problem  is  that  there 
are  often  many  possible  ways  to  resolve  each  conflict,  thus  contributing  to  an  explosion 
in  the  search  space.  More  bluntly,  resolving  conflicts  during  planning  multiplies  the 
exponential  task  of  conflict  resolution  by  the  exponential  task  of  planning. 

Both  of  these  alternatives  are  too  extreme.  Many  conflicts  are  simple  to  resolve  and 
can  be  delayed  until  the  end.  However,  those  that  are  tightly  interconnected  need  to  be 


14 


resolved  during  the  planning  process  (once  the  entire  group  of  conflicts  is  generated).  We 
have  developed  techniques  for  automatically  deciding  which  conflicts  should  be  resolved 
during  the  planning  process,  and  which  conflicts  should  be  delayed  until  planning  is  oth¬ 
erwise  complete.  The  basic  idea  is  to  analyze  the  potential  conflicts  between  operators 
in  the  operator  graph.  Roughly,  a  set  of  conflicts  can  be  postponed  if  we  can  show  in  the 
operator  graph  that  there  will  always  be  a  way  of  eliminating  the  conflicts  by  imposing 
partial  ordering  constraints  among  the  operators.  Under  these  circumstances,  the  post¬ 
poned  conflicts  can  be  ignored  during  planning,  and  can  always  be  resolved  by  imposing 
additional  ordering  constraints  on  the  otherwise  complete  plan.  The  technique  is  also 
recursive^  postponing  one  set  of  conflicts  may  then  allow  postponing  more  conflicts. 

In  loosely  coupled  domains,  many  of  the  conflicts  can  usually  be  postponed.  This  has 
the  effect  of  decoupling  the  planning  problem  into  two  pieces:  selection  of  the  operators 
for  achieving  the  goals,  and  ordering  the  operators  to  eliminate  conflicts  between  them. 
This  can  result  in  dramatic  reductions  in  the  amount  of  search  required  to  find  a  plan. 
More  complete  descriptions  of  this  technique  can  be  found  in  [43]  and  [44]. 

3,2.7  Conflict  resolution  strategies 

Not  all  conflicts  can  be  postponed.  In  this  case  the  question  arises  as  to  when  conflicts 
should  be  resolved  during  the  planning  process.  The  two  options,  resolve  conflicts  im¬ 
mediately,  and  resolve  conflicts  at  the  end,  represent  two  extreme  positions.  There  are 
several  other  interesting  options: 

DSEP  :  Wait  to  resolve  a  conflict  until  the  variable  bindings  guarantee  that  the  conflict 
will  occur. 

DUNF  :  Wait  to  resolve  a  conflict  until  there  is  only  a  single  way  of  resolving  it. 

DRES  :  Continue  checking  each  conflict  to  make  sure  it  can  be  resolved,  but  don  t 
resolve  conflicts  until  the  end. 

In  [38],  we  give  a  detailed  description  of  each  of  these  three  strategies,  and  show 
that  DSEP  always  generates  a  smaller  search  space  than  the  usual  strategy  of  resolving 
conflicts  as  soon  as  they  occur.  We  also  show  that  DUNF  generates  a  smaller  search 
space  than  DRES,  which  generates  a  smaller  search  space  than  the  strategy  of  delaying 
consideration  of  conflicts  until  the  very  end.  Furthermore,  strategies  DSEP  and  DUNF 
are  not  comparable:  DSEP  is  better  in  some  cases,  and  DUNF  is  better  in  others.  These 
theoretical  results  are  summarized  in  Figure  6 

These  results  have  led  to  a  more  complex  strategy  (DMIN)  [45]  that  is  a  combination 
of  both  DSEP  and  DUNF.  The  basic  idea  is: 

1.  Do  not  consider  any  threat  that  is  still  separable. 
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Figure  6:  Search  space  relationships  for  five  threat  removal  strategies. 

2.  Resolve  any  threat  that  has  only  one  possible  resolution. 

3.  Verify  that  there  is  a  way  of  resolving  any  remaining  threats  (not  covered  by  1  and 
2),  but  do  not  commit  to  any  particular  resolution  of  these  threats. 

We  have  shown,  theoretically,  that  this  combination  strategy  always  generates  a 
search  space  that  is  the  same  size  or  smaller  than  that  of  either  DSEP  or  DUNF. 
We  have  verified  all  of  these  theoretical  results  by  testing  these  strategies  on  several 
standard  planning  test  domains.  A  complete  description  of  these  experiments  can  be 
found  in  [38].  Joslin,  [27],  and  Kambhampati,  [28],  have  also  confirmed  several  of  these 
experimental  results. 


3.3  Partial  Plan  Evaluation 

In  general  we  want  planners  that  find  not  just  any  plan,  but  a  good  plan.  Given  utility 
models  for  a  planning  domain,  we  can  evaluate  and  compare  completed  plans.  However, 
it  is  computationally  intractable  to  generate  and  compare  all  possible  plans  for  a  signif¬ 
icant  domain.  Instead,  we  need  to  be  able  to  evaluate  and  compare  partial  plans  during 
the  planning  process,  so  that  planning  effort  can  be  focused  on  the  most  promising 
plans.  The  tough  part  here  is  being  able  to  estimate  the  cost  and  probability  of  suc¬ 
cess  for  those  parts  of  the  plan  that  are  not  yet  complete.  More  precisely,  we  want  to 
know  how  much  it  will  cost,  and  how  likely  it  is  that  we  can  achieve  the  remaining  open 
conditions  in  the  plan.  This,  together  with  the  actions  already  in  the  plan,  allows  an 
estimate  of  the  overall  worth  of  the  partial  plan. 

Operator  graphs  can  be  used  to  help  estimate  costs  and  probabilities  for  unfinished 
parts  of  the  plan.  To  see  how  this  works,  consider  the  operator  graph  in  Figure  7,  where 
Cl  through  C5  are  the  costs  of  operators  Oi  through  O5  respectively.  According  to  the 
operator  graph,  the  conditions  72,  S,  W,  t/,  and  V  can  all  be  satisfied  by  the  initial 
conditions.  As  a  result,  they  would  all  have  cost  0.  Next  consider  the  condition  A. 
There  are  two  possible  operators  that  can  be  used  to  achieve  A,  Oi,  and  02-  If  Oi  is 
used,  a  cost  of  Ci  will  be  incurred,  plus  the  cost  of  achieving  Oi’s  preconditions.  If  O2  is 
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Figure  7:  Propositional  operator  graph  with  operator  costs. 

used,  a  cost  of  C2  will  be  incurred,  plus  the  cost  of  achieving  02’s  preconditions.  Thus 
the  best  we  can  expect  for  achieving  A  is  the  minimum  of  these  two  costs,  min(Ci,  C2). 
Similarly,  the  best  we  can  expect  for  achieving  B  is  min(C3  +  C5,  C4). 

Now  consider  the  partial  plan  shown  in  Figure  8.  It  contains  one  step,  an  instance 
of  O25  and  two  open  conditions,  5,  and  B.  The  cost  of  O2  is  C2,  the  cost  for  S  is  0,  and 
the  best  cost  for  B  is  min(C3  +  Cs,  C4).  As  a  result,  the  best  cost  for  this  partial  plan  is 
C2  +  min(C'3  +  C5,  C4).  Using  this  cost,  this  plan  can  be  compared  with  other  candidate 
partial  plaits. 

In  the  above  example,  our  calculations  were  particularly  simple  because  the  operators 
are  all  propositional  (no  variables)  and  there  is  no  recursion  possible.  When  variables 
and/or  recursion  are  present,  the  analysis  becomes  much  more  complex.  For  example, 
when  variables  are  present,  the  cost  of  achieving  a  particular  open  condition  may  depend 
heavily  on  the  variable  bindings.  To  take  this  into  account  we  must  use  variable  analysis 
like  that  described  in  Section  3.2.3  to  find  out  the  possible  relevant  bindings  for  variables. 
We  then  compute  minimum  costs  for  these  different  possible  variable  bindings.  Recursion 
among  the  operators  poses  similar  problems. 

Although  we  only  considered  cost  in  this  example,  the  same  kind  of  calculations  can 
be  done  for  probability  of  success  and  utility.  The  key  is  that  the  operator  graph  allows 
us  to  consider  the  alternative  ways  of  achieving  each  potential  subgoal,  to  facilitate  some 
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Figure  8:  Partial  plan  with  two  open  conditions. 

estimate  of  how  costly  it  will  be  to  achieve  that  subgoal. 

A  more  detailed  description  of  this  ongoing  work  can  be  found  in  [41]. 


4  Plan  Evaluation 

For  plan  evaluation,  Rockwell  efforts  concentrated  on  the  building  of  utility  models,  and 
the  use  of  both  in-house  tools  (IDEAL  and  IDEAL-DEMOS  [48,  46])  and  commercial 
tools  (DEMOS  [51])  for  decision  making  and  inference.  The  first  significant  product 
of  this  effort  was  the  Transportation  Trade-off  Analyzer  (TTA).  This  was  a  decision- 
theoretic  based  tool  that  demonstrated  substantial  capability  enhancements  over  tra¬ 
ditional  transportation  schedulers  and  feasibility  estimators.  The  TTA  is  described  in 
Section  4.1  and  in  previous  annual  reports  to  ARPI  [4,  3]. 

The  TTA  was  demonstrated  at  the  end  of  the  first  year  of  the  program,  and  as  a 
result,  Rockwell  was  invited  to  apply  these  decision-theoretic  tools  to  the  crisis  action 
planning  domain  for  the  Third  Integrated  Feasibility  Demonstration  (IFD-3).  Rockwell, 
along  with  ISX,  developed  the  Course  Of  Action  Trade-off  Analyzer  (COATA)  for  Non- 
combatant  Evacuation  Operations.  The  functionality  provided  by  COATA  was  well 
received  by  the  Operational  Planning  Team  at  the  U.S.  Pacific  Command,  and  COATA 
was  showcased  in  the  original  version  of  TARGET  developed  for  IFD-3  as  one  of  only 
a  handful  of  Planning  Initiative  technologies  then  present  in  TARGET.  On  the  basis  of 
the  work  on  COATA,  Rockwell  was  invited  to  be  a  member  of  BBN’s  successful  bid  for 
the  Deployable  Joint  Task  Force  Advanced  Technology  Program.  COATA  is  described 
in  Section  4.3. 

Rockwell  made  available  two  decision-theoretic  tools  to  the  Planning  Initiative  com- 
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Figure  9:  Model  structure  used  in  the  Transportation  Trade-Off  Analyzer. 

munity  through  the  CPE.^  These  tools,  IDEAL  and  IDEAL-DEMOS,  are  general  decision- 
theoretic  modeling  tools  that  can  be  applied  to  a  wide  range  of  tasks.  IDEAL  is  currently 
in  use  by  hundreds  of  researchers  within  the  decision  theory  and  probabilistic  reasoning 
community.  Section  4.2  briefly  describes  the  capabilities  of  IDEAL  and  IDEAL-DEMOS; 
further  information  can  be  found  in  [47,  46]. 

Finally,  the  development  of  utility  models  for  crisis  management  pointed  to  the  need 
for  new  techniques  on  sensitivity  analysis,  specifically  on  the  computation  of  expected 
value  of  information.  The  new  techniques  and  algorithms  developed  by  Rockwell  for  this 
program  are  described  in  Section  4.4. 

^Rockwell  was  in  fact  one  of  the  first  PI  members  to  provide  tools  for  other  members. 
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4.1  Transportation  Trade-off  Analyzer 

As  the  ARPI  began  in  1991,  the  focus  was  on  increasing  the  capabilities  of  DOD  tools 
for  military  transportation  planning  and  scheduling.  Our  first  goal  was  to  explore  the 
domain  of  military  transportation  planning  and  to  capture  the  essential  uncertainties  and 
trade-offs  inherent  in  this  domain.  The  strategy  was  to  develop  utility  and  value  models 
that  express  trade-offs  between  consumption  of  resources  and  achievement  of  goals  under 
uncertainty,  in  order  to  express  the  preferences,  objectives,  and  knowledge  of  DOD 
planning  commands.  Once  this  was  done,  we  encoded  this  information  in  a  decision- 
theoretic  model  which  provided  help  with  such  capabilities  as  generating  alternative 
plans,  guiding  search,  and  controlling  computational  resource  allocation. 

The  domain  study  included  training  at  the  Armed  Forces  Staff  College.  The  resulting 
tool  was  named  “Transportation  Trade-off  Analyzer  (TTA)”.  Uncertainty  was  modeled 
explicitly  using  belief  networks  and  influence  diagrams  [32],  and  efficient  mechanisms 
for  reasoning  with  probabilistic  information  were  employed.  By  explicitly  modeling 
costs,  the  effects  of  delay,  and  uncertainties  in  the  transportation  process,  TTA  is  able 
to  optimize  the  transportation  plan  for  changing  requirements  and  under  the  changing 
conditions  of  the  real  world.  In  its  most  basic  configuration,  holding  costs  and  resources 
fixed,  the  TTA  model  performs  the  same  functionality  as  RAPIDSIM  or  DART  in  mea¬ 
suring  plan  feasibility.  By  relaxing  these  artificial  constraints,  TTA  is  able  to  provide 
significantly  enhanced  information.  The  characteristics  of  the  TTA  model  are  described 
in  depth  in  Rockwell’s  first  annual  report  [4].  A  picture  of  the  top  level  TTA  screen  is 
shown  in  Figure  9. 

4.2  Decision-Theoretic  Modeling  Tools  in  the  CPE 

After  the  first  year  of  the  PI  program,  Rockwell  made  two  general  decision-theoretic 
modeling  tools  available  to  the  other  members  of  the  PI  community.  These  tools,  IDEAL 
and  IDEAL-DEMOS  [48,  47,  46],  were  placed  in  the  Common  Prototyping  Environment 
(CPE),  and  were  some  of  the  first  researcher- based  tools  to  be  made  available  there. 

IDEAL  is  a  LISP-based  test  bed  for  work  in  influence  diagrams  and  Bayesian  net¬ 
works.  It  contains  various  inference  algorithms  for  belief  networks  and  evaluation  al¬ 
gorithms  for  influence  diagrams.  It  contains  facilities  for  creating  and  editing  influence 
diagrams  and  belief  networks  both  programmatically  and  via  a  CLIM-based  graphical 
editor  called  IDEAL-EDIT  [17].  An  IDEAL-EDIT  window  displaying  a  basic  IDEAL 
influence  diagram  is  shown  below  in  Figure  10. 

IDEAL-DEMOS  is  a  LISP  reimplementation  of  the  core  of  the  DEMOS  [51]  modeling 
language.  The  objective  was  to  complement  the  IDEAL  inference  system  with  some  of 
the  inference  capabilities  present  in  DEMOS. ^ 

^DEMOS  is  a  commercially  available  system  for  decision  modeling. 
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IDEAL  EDIT  Add  Type:  PROBABILITY-NODE  Algorithm:  Pearl  Clustering  File:  oil-wild-catter 


Add  New  Node;  M;  Deselect;  R:  Menu. 


Copyright  by  Rockwell  Internationsl  Corporstion  as  an  unpublished 
work.  This  work  was  created  1989-1992  and  contains  proprietary 
inforaation  of  Rockwell  International  Corporation.  All  rights  are 
reserved.  Pisclosure  without  written  authoriration  is  prohibited 


Figure  10:  An  example  of  an  influence  diagram  in  IDEAL-EDIT. 
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Figure  11:  The  graph  describes  the  buildup  of  U.S.  citizens  at  an  assembly  area. 

4.3  Course  Of  Action  Trade-off  Analyzer 

The  success  of  the  TTA  model  and  the  knowledge  learned  from  its  construction  was 
beneficial  in  several  ways.  As  well  as  providing  feedback  that  was  useful  in  other  research 
efforts,  Rockwell  was  invited  to  apply  the  same  techniques  used  in  the  TTA  model  to 
the  domain  of  crisis  action  planning  for  the  Third  Integrated  Feasibility  Demonstration 
(IFD-3)  of  the  ARPA/Rome  Labs  Planning  Initiative.  Rockwell  provided  the  decision- 
theoretic  plan  evaluation  capabilities  for  the  Theater-level  Analysis,  Replanning  and 
Graphical  Execution  Toolbox  (TARGET)  during  IFD-3.  These  capabilities  included  the 
means  of  estimating  the  feasibility,  costs,  casualties,  and  times  associated  with  various 
courses  of  actions  (COA).  The  targeted  user  was  the  executive  officer  of  the  Opera¬ 
tional  Planning  Team  (OPT).  The  objective  was  to  provide  a  computational  tool,  called 
“Course  of  Action  Trade-off  Analyzer”  (COATA),  for  capturing  the  assumptions  and  the 
rationale  behind  the  planning  process.  The  output  of  COATA  is  a  set  of  estimates  and 
entries  that  can  support  the  development  of  a  COA  selection  matrix.  Figure  11  shows 
an  example  of  these  outputs. 

COATA  consists  of  three  components:  a  decision  model,  an  inference  engine,  and  a 
graphical  interface.  The  decision  model  was  developed  by  Rockwell  personnel  in  con¬ 
junction  with  U.S.  Pacific  Command  to  support  COA  analysis  for  Non-combatant  Evac- 
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Figure  12:  Top  level  model  for  the  NEO  course  of  action  analysis  tool.  The  tool  generates 
graphs  describing  risks  for  U.S.  citizens  and  estimates  of  transportation  times. 
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uation  Operations  (NEO).  The  inference  engine  is  part  of  a  Rockwell  in-house  tool  for 
decision  making  under  uncertainty,  and  the  run-time  graphical  interface  was  developed 
primarily  by  ISX. 

The  model  assumes  the  following  scenario:  A  situation  arises  which  requires  the 
evacuation  of  U.S.  civilians  from  a  foreign  country.  These  U.S.  citizens,  located  in  one 
or  more  regions  in  the  host  country,  are  told  to  move  from  their  current  locations  to 
one  or  more  assembly  areas  in  the  country.  Simultaneously,  U.S.  military  forces  are 
deployed  from  their  current  locations  in  the  world  to  the  assembly  areas.  Upon  arrival, 
the  military  forces  secure  the  assembly  areas  for  civilian  transit.  Transportation  assets 
are  utilized  to  move  the  U.S.  citizens  from  the  assembly  areas  to  safe  haven(s),  which 
are  generally  located  outside  of  the  country.  The  top  level  model  (in  the  form  of  an 
influence  diagram)  is  shown  in  Figure  12. 

The  COATA  model  allows  the  user  to  instantiate  a  generic  NEO  plan  with  specific 
locations,  forces,  and  destinations,  along  with  the  (possible)  uncertainties  associated 
with  this  information  (represented  by  probability  distributions),  and  then  to  perform 
trade-off  analyses  for  different  courses  of  action.  The  user  or  some  other  automated 
planner  must  specify  the  specific  data  regarding  country  locations  and  number  of  citizens, 
the  assembly  areas,  safe  havens,  security  forces,  and  transportation  assets.  In  addition, 
estimates  on  the  risk  to  the  U.S.  citizens  and  the  military  of  attrition  (death)  at  various 
stages  of  the  operation  are  also  needed.  The  different  COAs  are  defined  by  the  choice 
of  security  forces  (including  their  original  location,  date  of  availability  and  capabilities), 
safe  havens,  and  transportation  assets.  Note  that  the  use  of  probability  distributions 
allows  the  specification  of  uncertain  and  incomplete  data  that  can  be  refined  during  the 
various  stages  of  the  planning  process. 

Once  the  necessary  information  is  specified,  the  model  performs  a  dynamic  simulation 
of  the  flow  of  U.S.  citizens  from  their  initial  locations  to  assembly  areas  and  then  to 
the  safe  havens.  Some  of  the  more  important  outputs  of  the  model  include:  time  to 
completion,  total  of  U.S.  civilian  and  military  casualties,  speed  of  first  response,  and 
maximum  build-up  of  U.S.  citizens  at  assembly  areas.  The  model  also  includes  risk 
factors  associated  with  both  U.S.  citizens  and  U.S.  military  personnel  as  a  function  of 
time.  These  risk  factors  are  used  to  calculate  expected  casualties  for  various  options. 
Additionally,  since  the  uncertainty  of  any  input  is  represented  in  the  model,  the  tool  is 
capable  of  generating  an  expected  value  for  the  parameter  as  well  as  a  full  uncertainty 
distribution  for  any  of  the  output  parameters. 

The  COATA  tool  provided  a  successful  demonstration  of  technology  transfer  for 
IFD-3  at  PACOM  in  May  and  September  of  1993.  The  senior  officer  for  the  OPT  at 
PACOM,  who  helped  to  develop  the  model,  gave  high  marks  to  the  capabilities  of  the 
tool.  Currently,  the  Rockwell  Palo  Alto  Laboratory  is  a  member  of  an  on-going  project 
to  extend  the  TARGET  capabilities  to  the  deployable  Joint  Task  Force.  The  second 
annual  report  contains  more  specific  information  about  COATA  [3]. 


24 


4.4  Sensitivity  Analysis  in  Planning 

The  experience  in  building  computer-based  utility  models  for  evaluating  and  simulating 
crisis  scenarios  pointed  towards  the  need  for  more  sophisticated  techniques  and  methods 
for  sensitivity  analysis.  It  is  necessary  to  decide  which  aspects  of  a  scenario  are  essential 
for  planning  and  need  to  be  represented  in  the  model,  and  which  aspects  can  be  ignored. 
The  reason  is  simple:  given  that  information  is  an  expensive  commodity,  every  model 
must  balance  the  desire  for  maximum  precision  and  detail  against  the  need  to  get  the  job 
done  with  limited  time,  data,  and  computational  resources.  This  tension  is  particularly 
acute  in  military  crisis  planning,  where  the  stakes,  risks  and  complexities  are  typically 
high,  and  where  the  deadlines  are  always  tight. 

Decision  theory,  and  the  techniques  of  decision  analysis  that  are  based  on  it,  provide 
a  number  of  powerful  methods  for  analyzing  these  trade-offs  between  the  precision  of  a 
model  and  the  resources  required  to  analyze  it.  The  explicit  representation  of  uncertainty 
in  the  form  of  probability  distributions  allows  comparison  of  the  effects  of  the  different 
sources  of  uncertainty,  and  analysis  of  the  costs  and  benefits  of  obtaining  more  detailed 
information  to  reduce  the  uncertainty.  A  planner  needs  to  know  which  uncertainties 
matter,  but  he  or  she  also  needs  to  know  how  to  allocate  economic  resources  to  resolve 
or  reduce  those  uncertainties. 

We  have  therefore  focused  on  giving  planners  tools  for  sensitivity  analysis  and 
scenario  management.  Faced  with  dozens,  possibly  hundreds,  of  variables,  with  com¬ 
plex  interdependencies  and  interactions  among  them,  planners  need  principled,  efficient 
techniques  for  identifying  the  most  relevant  or  essential  factors.  They  need  to  know 
where  their  strategies  are  most  likely  to  ’’break,”  and,  in  particular,  what  uncertainties 
contribute  most  to  the  fragility  of  their  plans.  Such  knowledge,  in  turn,  helps  them  allo¬ 
cate  resources  (e.g.,  deployment  of  intelligence  assets,  consultation  with  other  experts) 
for  information-gathering  and  uncertainty/risk  reduction. 

For  example,  consider  the  NEO-COA  model,  developed  by  the  Rockwell  Palo  Alto 
Laboratory,  in  collaboration  with  planners  from  PACOM,  the  Pacific  Crisis  Planning 
team  introduced  in  Section  4.3,  and  described  in  [3].  NEO-COA  was  designed  to  help 
military  planners  analyze  alternate  courses  of  action  for  the  evacuation  of  U.S.  citizens 
from  crisis  areas.  Examples  of  uncertainties  explicitly  represented  in  NEO-COA  are  the 
numbers  of  U.S.  citizens,  their  locations  in  particular  crisis  regions,  the  accumulated 
risks  they  face  over  time,  and  the  rates  at  which  they  move  to  assembly  areas  and 
points  of  embarcation.  The  number  of  citizens  in  the  capital  might  be  a  probability 
distribution  with  median  500,  and  a  90%  probability  that  the  total  is  between  300  and 
1000.  Should  planners  go  ahead  now  and  schedule  air  transports  with  capacity  for,  say, 
1000  evacuees?  Should  they  wait  until  intelligence  provides  a  more  precise  estimate?  Or 
should  they  schedule  air  transports  to  carry  500  evacuees,  with  further  capacity  500  to 
be  held  in  reserve?  NEO-COA  can  help  planners  compare  these  alternatives,  and  assess 
the  expected  value  of  better  information  relative  to  the  costs  of  waiting  for  it. 
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4.4.1  The  Expected  Value  of  Information 

The  Expected  Value  of  Information  (EVI)  is  the  best  sensitivity  measure  because  it 
analyzes  a  variable’s  importance  in  terms  of  the  recommended  action,  and  it  expresses 
that  importance  in  the  value  units  of  the  problem.  Other  methods,  such  as  rank-order 
correlation,  measure  a  variable’s  contribution  to  the  overall  uncertainty  in  the  model’s 
outputs,  expressed  as  a  probability  correlation.  EVI,  in  contrast,  measures  a  variable  s 
importance  in  terms  of  what  matters  —  in  the  case  of  NEO-COA,  for  example,  civilian 
lives. 

Certain  methods,  such  as  deterministic  perturbation,  measure  a  variable’s  impor¬ 
tance  in  terms  of  what  matters;  yet  they  can  still  mislead.  For  example,  suppose  the 
only  available  evacuation  transport  is  a  ship  that  can  hold  1500  passengers,  and  the  key 
plan  option  is  the  choice  of  safe  haven.  The  number  of  U.S.  citizens  at  risk  in  the  crisis 
region  may  affect  the  overall  outcome  of  the  operation,  but  will  probably  not  affect  the 
plan  choice.  EVI  would  reveal  uncertainty  about  the  number  of  evacuees  to  be  irrelevant 
to  the  plan  choice,  while  deterministic  perturbation  might  show  high  sensitivity  to  this 
variable. 

The  use  of  less  powerful  sensitivity  measures  can  mislead  planners  into  thinking  that 
it  is  worthwhile  to  wait  for  information  about  variables  which  are,  in  fact,  irrelevant. 
Such  sensitivities  reflect  the  potential  of  new  information  to  change  beliefs  or  values 
on  outcomes.  EVI,  in  contrast,  measures  the  potential  of  new  information  to  change 
decisions. 

4.4.2  Computing  the  Value  of  Information  in  Large  Decision  Models 

To  date,  calculation  of  EVI  has  remained  largely  intractable  for  models  of  any  reasonable 
size.  In  models  with  discrete  variables  (i.e.,  models  that  admit  solutions  using  decision 
trees),  EVI  has  computational  complexity  that  is  exponential  in  the  number  of  state 
variables.  For  models  with  continuous  variables  such  as  NEO-COA,  there  were,  to  our 
knowledge,  no  algorithms  —  exponential  or  otherwise  —  for  calculating  EVI.  Thus,  an 
important  part  of  our  work  has  been  to  develop  efficient  methods  for  calculating  EVI  in 
probabilistic  models  for  planning. 

The  innovative  approach  we  developed  is  based  on  three  features:  The  synthesis  of 
an  initial  high  dimensional  problem  into  a  summarizing  function,  the  introduction  of 
special  information  measures,  and  an  efficient  pre-posterior  analysis.  The  algorithmic 
techniques  are  conceptualized  in  Figure  13,  and  a  precise  description  of  the  algorithm 
and  the  results  of  its  application  to  NEO  can  be  found  in  [6,  7]. 

D  denotes  a  set  of  m  mutually  exclusive  actions;  A"  is  a  vector  of  n  state  variables, 
all  of  which  may  be  defined  as  probability  distributions.  The  set-up  alone  makes  clear 
why  the  calculation  of  EVI  is  difficult:  it  is  a  high-dimensional  problem;  worse,  each 
dimension  is  itself  a  random  variable.  We  assume  a  value  function  V{d,X),  which 
specifies  value  to  the  decision  maker  when  X  assumes  a  particular  value  and  action  d 
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Figure  13:  Conceptual  overview  of  the  algorithm  for  estimating  EVI. 

has  been  taken.  Formally,  the  definition  of  EVI  is  as  follows: 

EVI  =  maxF;[V(d,  V)|X,s]  -  m&x  E[V{d,  X)\s]  (1) 

d  d 

where  E  denotes  expectation  and  s  stands  for  the  decision  maker’s  prior  state  of  infor¬ 
mation.  (We  include  s  simply  to  emphasize  that  all  the  quantities  in  the  model  have 
been  assessed  relative  to  the  decision  maker’s  prior  state  of  information.)  That  is,  EVI  is 
the  difference  in  expected  value  achieved  by  taking  the  optimal  action  with  information 
on  X,  and  the  optimal  action  without  information  on  X.  For  simplicity,  in  what  follows 
we  assume  that  D  consists  of  just  two  discrete  actions,  d\  and  0^2 • 

Observing  Figure  13,  we  see  that  X  and  D  are  applied  as  inputs  to  a  linearizing 
method  to  be  described  shortly;  the  output  is  a  variable  we  call  Z.  Z  is  defined  as  the 
difference  in  value  between  ^2  and  d\,  assuming  that  d2  is  the  preferied  action  (i.e.,  d^ 
yields  higher  expected  value  than  di). 

Z  =  V{X,d2)-V{X,d^).  (2) 

In  Figure  14,  we  show  a  graph  of  the  probability  distribution  for  Z. 

Z  is  the  pivotal  element  of  our  analysis  because  it  essentially  collapses  the  dimen¬ 
sionality  of  our  original  problem.  We  begin  with  n  probability  distributions  foi  each 
of  the  variables  of  X,  a  separate  set  of  actions  D,  and  a  value  function  V{X,d).  The 
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Figure  14;  Probability  distribution  on  Z . 

probability  distribution  on  Z  —  a  simpler,  two-dimensional  quantity  —  encodes  all  the 
information  from  these  elements  that  is  vital  for  calculating  EVl. 

For  the  next  portion  of  the  analysis  we  consider  the  impact  of  evidence  or  information 
e  on  any  of  the  state  variables  in  X.  We  use  two  measures,  ESS  (equivalent  sample  size) 
and  RIM  (relative  information  multiple)  to  describe  the  magnitude  of  new  information. 
ESS  is  commonly  used  in  statistical  decision  theory;  RIM  is  a  new  measure  we  have 
introduced.  If  X  is  the  variable  about  which  we  expect  to  receive  new  information, 
then  ESS  is  an  equivalent  number  of  observations  on  A^’s  value,  while  RIM  specifies  a 
reduction  factor  in  the  variance  of  the  probability  distribution  for  X.  The  advantage  of 
a  RIM  is  that  it  is  defined  in  terms  of  a  parameter  already  familiar  to  the  decision  maker 
—  the  variance  of  the  probability  distribution  for  X.  In  essence,  these  measures  allow 
the  user  to  calculate  the  value  of  gathering  more  information  about  a  certain  variable, 
where  “more”  is  formalized  as  a  multiplicative  factor  of  the  initial  state  of  uncertainty: 
e.g.,  what  would  be  the  value  of  decreasing  by  a  factor  of  2  our  uncertainty  on  the  exact 
number  of  citizens  in  the  capital? 

The  final  step  of  the  analysis  requires  preposterior  analysis  on  Z  to  calculate  the 
effects  of  e,  expressed  in  RIM’s  or  ESS,  on  combinations  of  state  variables.  We  rely 
on  our  linearizing  method,  probabilistic  analysis,  and  numerical  techniques  to  solve  for 
EVL 

We  have  applied  the  algorithm  to  a  large-scale  decision  model  for  planning  NEO-COA 
introduced  in  Section  4.3.  The  results  are  described  in  [6,  7],  and  include: 

•  an  operational  framework  for  calculating  EVl  based  on  ideas  from  statistical  deci- 
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sion  theory; 

•  an  efficient  approximation  algorithm  for  EVI; 

•  a  formal  analysis  which  guarantees  the  convergence  of  the  algorithm,  using  a  stop¬ 
ping  rule  which  tests  a  simple  condition  involving  pre-specified  error  parameters 
and  the  algorithm’s  current  estimate;  in  addition,  a  guarantee  that  the  algorithm’s 
estimate  is  ’’probably  approximately  correct”  —  that  is,  with  probability  at  least 
1  —  the  algorithm’s  output  is  within  relative  error  e  of  the  ’’true”  answer. 

•  a  knowledge  representation  for  evidence  or  information  that  accommodates  per¬ 
fect/partial  EVI; 

•  an  application  of  the  algorithm  to  the  NEO-COA  model;  and 

•  implementation  of  the  algorithm  in  detachable  software  modules  using  DEMOS  [51], 
a  commercially  available  software  tool. 

In  addition,  we  have  recently  completed  a  set  of  experiments  demonstrating  that 
the  algorithm  is  stable  in  the  sense  that,  as  its  running  time  increases,  its  estimates 
converge  at  roughly  the  same  rate  as  answers  calculated  using  exact  techniques.  We 
have  verified  that,  while  the  linearizing  assumption  introduces  error,  its  errors  are  at 
least  well-behaved  compared  to  the  errors  introduced  by  any  estimation  algorithm.  This 
is  important,  because  we  want  to  insure  that  the  errors  in  the  algorithm’s  outputs  are 
at  least  systematic,  and  that  they  decrease  monotonically  as  we  increase  the  algorithm’s 
running  time. 

Our  immediate  goal  is  to  apply  the  resulting  algorithm  to  other  large  probabilistic 
models  for  planning.  We  are  especially  interested  to  see  how  well  it  ’’scales  up”  in  large, 
temporal  models  for  planning  under  uncertainty,  though  we  do  not  anticipate  difficulties 
with  such  an  application. 

In  sum,  we  have  developed,  implemented  and  tested  an  efficient  algorithm  for 
anytime  approximation  of  EVI  in  large  probabilistic  models,  for  perfect  and/or 
partial  information  on  any  combination  of  variables,  and  with  no  limiting  assumptions 
about  the  distributions  of  the  variables,  the  nature  of  the  value  function,  or  the  number 
of  possible  actions. 


5  Plan  Simulation 

Our  work  on  plan  simulation  was  directed  towards  creating  a  tool  for  supporting  the 
functionality  portrayed  in  Figure  15.  Here,  a  human  planner  is  confronted  with  a  sit¬ 
uation,  an  objective  and  some  courses  of  action  (plans)  that  are  believed  to  achieve 
the  objective.  The  human  planner,  however,  is  looking  for  assistance  in  simulating  the 
behavior  of  each  of  these  plans  to  make  a  choice  on  which  one  to  adopt.  The  purpose  of 
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Figure  15;  Schematic  view  of  a  plan  simulatioji  scenario. 

our  work  under  this  part  of  the  contract  is  to  develop  such  a  tool,  which  is  comprised  of 
two  major  components: 

1.  A  language  for  modeling  the  situation  at  hand. 

2.  A  set  of  algorithms  that  operate  on  the  model  in  order  to  simulate  a  plan. 

These  two  components  are  described  in  the  following  two  sections  (Sections  5.1  and  5.2. 


5.1  Action  Networks 

The  first  step  in  simulating  and  analyzing  a  plan  is  to  describe  the  domain  in  which 
this  plan  will  be  executed.  According  to  the  terminology  used  in  SWAT  team  B  report, 
this  amounts  to  creating  a  situation  model.  In  our  research  program,  we  proposed  the 


30 


formalism  of  action  networks  to  construct  a  situation  model  and  used  them  to  simulate 
military  plans  (see  Figure  16).  In  the  remainder  of  this  section,  we  will  focus  on  the 

following: 

1.  Provide  a  brief  introduction  into  action  networks  as  a  domain  modeling  language 

2.  Summarize  the  current  status  of  action  networks  based  on  research  conducted 
under  this  contract 

5.1.1  Introduction  into  action  networks 

An  action  network  is  a  modeling  tool  oriented  mainly  towards  dynamic  domains  that  can 
be  controlled  by  agents.  Action  networks  are  very  closely  connected  to  what  is  known 
as  causal  networks,  which  are  the  most  sophisticated  and  effective  tools  for  modeling 
domains  under  uncertainty  as  demonstrated  by  many  real-world  applications  [5,  32]. 

The  difference  between  (the  recently  developed)  action  networks  and  (the  by  now 
matured)  causal  networks  is  that  causal  networks  do  not  include  explicit  constructs  for 
modeling  time  and  action,  which  are  the  key  primitives  for  generating  plans  in  dynamic 
domains.  Our  work  on  action  networks  was  aimed  mainly  on  introducing  these  two 
primitives  to  causal  networks.  We  emphasize  here  that  each  action  network  is  equivalent 
to  a  causal  network.  Therefore,  action  networks  are  supported  by  all  the  theoretical 
and  practical  tools  that  have  been  developed  for  causal  networks,  including  the  theory 
developed  in  [32]  and  the  various  commercial  tools  available  from  a  number  of  vendors. 

5.1.2  Progress  on  Action  Networks 

The  first  publication  on  action  networks  appeared  in  [14],  which  describes  the  following 
additions  that  action  networks  brought  to  causal  networks; 

1.  Contvollable  variables  (action),  which  are  variables  in  a  causal  structure  that  can 
be  controlled  by  agents.  With  respect  to  a  controllable  variable,  an  action  net¬ 
work  may  designate  a  number  of  other  variables  that  determine  the  conditions  of 
controllability.  These  are  called  precondition  variables  and  are  connected  to  the 
controllable  variables  using  a  precondition  arc. 

2.  Persistent  variables  (facts),  which  are  variables  in  a  causal  structure  the  values  of 
which  persists  over  time  unless  acted  upon  by  agents. 

Variables  that  are  neither  actions  nor  facts  are  called  events.  These  are  variables  that 
cannot  be  controlled  but  at  the  same  time  will  retain  their  values  over  time. 

In  terms  of  implementational  details,  action  networks  and  a  number  of  corresponding 
inference  algorithms  are  implemented  in  C-NETS  [11],  a  common  lisp  system  of  which 
we  are  constructing  a  C  version. 
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Figure  16:  An  example  of  an  action  network. 
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In  addition  to  controllable  and  persistent  variables,  action  networks  have  brought 
to  traditional  causal  networks  the  ability  do  symbolic  and  qualitative  reasoning  under 
uncertainty.  Traditionally,  causal  networks  have  been  probabilistic  in  the  sense  that 
cause-effect  interactions  are  quantified  by  providing  probabilities  of  effects  given  their 
causes.  But  action  networks  are  not  necessarily  probabilistic.  Instead,  a  causal  network 
will  consist  of  two  parts:  a  directed  graph  T  representing  a  “blueprint”  of  the  causal  rela¬ 
tionships  in  the  domain  and  a  quantification  Q  of  these  relationships.  The  quantification 
Q  introduces  a  representation  of  the  uncertainty  in  the  domain  because  it  specifies  the 
degree  to  which  causes  will  bring  about  their  effects.  Action  networks  allow  uncertainty 
to  be  specified  at  different  levels  of  abstraction: 

1.  Point  probabilities,  which  is  the  common  practice  in  causal  networks  [32]. 

2.  Order— of— magnitude  probabilities,  also  known  as  epsilon  probabilities  [23,  24,  15]. 

One  should  emphasize  here  that  causal  networks  are  usually  associated  with  proba¬ 
bilistic  reasoning  and  therefore  are  perceived  to  inherit  the  problems  associated  with  this 
mode  of  reasoning,  especially  the  infamous  where-do-the-numbers-come-from?  problem. 
We  stress,  however,  that  causal  networks  upon  which  action  networks  are  based  could 
be  purely  symbolic  if  the  user  so  desires  [16].  The  principal  investigators  in  this  program 
have  been  leading  the  efforts  for  going  beyond  point  probabilities  in  quantifying  causal 
networks;  their  work  on  this  can  be  found  elsewhere  [9,  23]. 

Although  action  networks  support  order-of-magnitude  probabilities,  they  do  not  sup¬ 
port  pure  symbolic  modeling  yet.  We  plan  to  address  this  ability,  however,  in  future 
research. 

5.2  Algorithms 

5.2.1  Introduction 

In  our  approach,  plan  simulation  reduces  computationally  to  reasoning  with  a  belief 
network.  There  is  a  number  of  algorithms  in  the  literature  for  this  purpose  [32],  but  they 
do  perform  poorly  on  networks  generated  for  plan  evaluation.  The  basic  problem  is  the 
existence  of  undirected  cycles  (loops)  in  the  network,  which  complicates  the  computation 
process.^  Networks  generated  for  plan  evaluation  tend  to  have  many  loops  given  their 
temporal  nature.  For  example,  a  network  with  one  loop  could  have  more  than  twenty 
loops  when  expanded  over  three  time  steps. 

To  deal  with  networks  containing  loops,  researchers  have  appealed  to  two  concepts: 
clustering  and  conditioning.  In  clustering,  the  network  is  aggregated  to  allow  the  de¬ 
composition  of  computations  into  simpler  ones.  In  conditioning,  certain  variables  are 
instantiated  to  allow  for  conditional  independences  to  hold,  thus  leading  to  more  efficient 

no  loops  exist,  the  network  is  singly-connected  and  the  computational  complexity  is  linear  in  the 
number  of  nodes  and  arcs. 
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computations.  The  method  of  clustering  is  due  to  Lauritzen  and  Spiegelhalter  [30],  and 
the  conditioning  algorithm  is  due  to  Pearl  [32]. 

As  was  observed  by  Pearl  [32],  the  use  of  conditioning  to  facilitate  the  computation 
of  beliefs  is  not  foreign  to  human  reasoning.  When  we  find  it  hard  to  assess  our  belief 
in  a  proposition,  we  often  make  hypothetical  assumptions  that  render  the  assessment 
simpler.  This  correspondence  to  human  reasoning  is  probably  the  reason  why  cutset 
conditioning  was  first  expected  to  be  more  efficient  than  other  methods  for  evaluating 
belief  networks.  But  as  observed  later  in  [33],  practice  has  shown  that  conditioning 
turned  out  to  be  computationally  less  attractive  than  the  Lauritzen  and  Spiegelhalter 

algorithm  [30]. 

Our  commitment  to  conditioning  methods  stems  from  two  factors:  1)  We  know^that 
we  can  apply  these  methods  to  any  network  independently  of  its  quantification  [O],”*  and 
2)  we  have  identified  a  number  of  computational  techniques  that  are  best  embedded 
in  the  context  of  a  conditioning  method.  To  provide  a  set-up  for  implementing  these 
techniques,  we  developed  the  method  of  dynamic  conditioning,  which  is  a  refinement  of 
cutset  conditioning  that  makes  it  comparable  in  computational  performance  to  cluster¬ 
ing  methods.  Based  on  dynamic  conditioning,  we  developed  the  method  of  e-bounded 
conditioning,  which  is  an  approximate  method  that  allows  one  to  trade  efficiency  foi 
accuracy  of  computation.  We  also  investigated  two  specialized  algorithms.  The  first 
is  a  prediction  algorithm  that  applies  to  action  networks  only  by  taking  advantage  of 
the  recurrent  structure  of  the  temporal  expansion  of  the  action  network.  The  second 
algorithm  is  also  for  prediction  and  is  applied  to  order-of-magnitude  networks  only  by 
taking  advantage  of  approximating  probabilistic  uncertainty  to  orders-of-magnitude  un¬ 
certainty.  We  will  discuss  each  of  these  algorithms  in  the  following  sections.  But  we 
first  summarize  the  algorithms  according  to  the  properties  that  make  them  of  interest 
to  plan  simulation: 

Dynamic  Conditioning:  A  method  for  the  exact  simulation  of  plans  and  is  based 
on  hypothetical  reasoning  (reasoning  by  cases).  The  significance  of  the  algorithm 
stems  from:  (1)  It  is  the  basis  for  the  algorithm  of  e-bounded  conditioning  to 
be  discussed  next;  and  (2)  It  is  an  infra-structure  for  realizing  computational 
techniques  that  are  based  on  reasoning  by  cases. 

e-bounded  conditioning:  A  method  for  the  approximate  simulation  of  plans  which 
allows  one  to  trade  the  exactness  of  the  simulation  with  the  time  it  takes  to  com¬ 
pute  the  simulation.  The  significance  of  this  algorithm  stems  from  the  following 
observation:  Although  the  results  of  a  particular  simulation  may  not  be  exact, 
they  may  be  enough  to  support  a  specific  judgment  about  some  plan. 

Action  Network  Predict:  An  algorithm  for  predicting  the  effect  of  plans  with  respect 
to  a  subclass  of  action  networks.  The  algorithm  has  a  polynomial  computational 
complexity  on  this  subclass. 

^To  the  best  of  our  knowledge,  there  are  no  current  results  (or  implementation)  that  suggests  that 
clustering  methods  can  work  for  quantifications  other  than  probabilistic. 
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Figure  17:  Current  realization  of  the  Plan  Simulator  and  Analyzer  (PSA)  tool. 

Kappa  Predict:  An  algorithm  for  predicting  the  effect  of  plans  with  respect  to  order- 
of-magnitude  action  networks.  The  algorithm  has  a  polynomial  computational 
complexity  but  may  be  unable  to  provide  complete  answers  in  certain  cases.  The 
significance  of  this  algorithm  stems  from  at  least  two  factors:  (1)  Although  the 
answers  it  provides  may  be  incomplete,  they  may  be  enough  to  judge  the  quality 
of  given  plans;  and  (2)  It  is  a  major  building  block  of  e-bounded  conditioning  that 
we  discussed  earlier. 

Figure  17  shows  a  refinement  of  the  schematics  in  Figure  15,  with  a  precise  charac¬ 
terization  of  inputs  and  outputs  including  a  choice  of  the  algorithms  described  above  for 
computation. 

5.2.2  Dynamic  Conditioning 

As  a  first  step  in  our  research  on  algorithms,  we  have  attempted  to  show  that  condi¬ 
tioning  methods  can  be  as  efficient  as  clustering  methods,  thus  setting  the  ground  for 
our  further  improvements  on  this  approach.  In  particular,  we  refined  cutset  condition¬ 
ing  into  the  method  of  dynamic  conditionining  [12],  which  was  shown  experimentally  to 
perform  comparably  to  the  Lauritzen  and  Spiegelhalter  clustering  method. 

The  breakthrough  that  permitted  our  results  is  the  discovery  of  two  important  no¬ 
tions  in  connection  to  conditioning  methods:  local  and  relevant  cutsets,  which  are  subsets 
of  a  loop  cutset  on  which  the  method  of  conditioning  is  based.  Relevant  cutsets  can  be 
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identified  in  linear  time  and  they  usually  lead  to  exponential  savings  when  computing 
beliefs  in  the  light  of  assumptions.  Local  cutsets,  on  the  other  hand,  eliminate  the  need 
for  considering  an  exponential  number  of  assumptions  because  they  identify  assumptions 
that  lead  to  the  same  beliefs.  Local  cutsets  can  be  computed  in  polynomial  time  from 
relevant  cutsets.  The  algorithm  of  dynamic  conditioning  is  a  refinement  of  cutset  con¬ 
ditioning  utilizing  the  notions  of  relevant  and  local  cutsets.  We  provided  experimental 
results  in  [12]  showing  that  dynamic  conditioning  performs  comparably  to  the  Lauritzen 
and  Spiegelhalter  algorithm  as  implemented  in  IDEAL  [19,  48].  We  also  discussed  net¬ 
work  structures  on  which  dynamic  conditioning  has  a  linear  computational  complexity, 
while  cutset  conditioning  leads  to  an  exponential  behavior. 

Dynamic  conditioning  has  been  implemented  in  C-NETS  [11]®  and  tested  for  cor¬ 
rectness  against  IDEAL  [48]  over  hundreds  of  randomly  generated  networks. 

5.2.3  Prediction  Algorithm:  Specific  to  action  networks 

Temporal  causal  networks  that  result  from  expanding  atemporal  networks  (using  the 
suppressor  model)  [14]  tend  to  have  a  large  number  of  undirected  cycles.  These  cycles 
lead  to  networks  that  are  difficult  to  manage  computationally;  in  fact,  inference  in  such 
networks  is  known  to  be  NP-hard. 

To  deal  with  such  networks,  we  have  been  exploring  a  number  of  approaches.  One  of 
these  approaches  tries  to  prove  independence  properties  about  temporal  networks  that 
are  then  utilized  by  algorithms  in  decomposing  complex  computations  into  simpler  ones 
that  can  be  performed  in  parallel.  A  second  approach  explores  task-specific  procedures 
with  computation  of  approximate  answers  sacrificing  accuracy  in  order  to  gain  speed. 

A  key  result  obtained  under  this  investigation  is  that  temporal  networks  that  result 
from  expanding  singly  connected  atemporal  networks  (without  cycles)  have  some  strong 
independences  properties.  In  particular,  it  can  be  shown  that  the  parents  7r(Aj)  of  any 
node  Nt  at  time  t  become  independent  given  the  images  of  Nt  and  7r(Ai)  at  time  t  —  1. 
This  independence  is  strong  enough  to  justify  the  derivation  of  a  sound  and  complete 
algorithm  for  prediction  that  is  linear  in  the  number  of  time  steps  and  exponential 
only  in  twice  the  number  of  parents  per  node.  The  algorithm  is  valid  for  inference  in 
probabilistic,  kappa,  and  symbolic  causal  networks. 

We  stress  here  that  although  the  original  atemporal  network  may  be  singly  connected, 
its  temporal  expansion  may  have  many  cycles,  which  makes  it  very  difficult  for  general 
purpose  inference  algorithms.  But  by  virtue  of  the  expansion  method  (according  to  the 
suppressor  model),  the  resulting  multiply  connected  network  is  guaranteed  to  satisfy 
some  strong  independence  properties. 

®C-NETS  is  an  environment  for  representing  and  computing  with  generalized  causal  networks  [9,  13]. 
Therefore,  dynamic  conditioning  is  not  restricted  to  reaisoning  with  probabilistic  causal  networks,  but 
is  also  applicable  to  other  classes  of  networks,  such  as  kappa  causal  networks  [21]  and  symbolic  causal 
networks  [10]. 
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5.2.4  Prediction  Algorithm:  Specific  to  order-of-magnitude  networks 

The  second  approach  we  investigated  is  an  algorithm  for  prediction  that  takes  as  input 
a  network  quantified  with  kappas  (degrees  of  plausibility  based  on  probabilities)  and 
returns  as  output  the  most  plausible  state  of  each  variable. 

The  algorithm  has  the  following  properties: 

1.  The  asymptotic  complexity  is  0(max[£,  N  x  LU])  where  E  is  the  number  of  edges 
in  the  network,  N  is  the  number  of  nodes,  and  LU  are  table  look-up  operations. 

2.  The  algorithm  is  sound:  If  the  algorithm  predicts  that  x  is  the  most  plausible  state 
of  X,  then  the  probability  of  A  =  a;  is  arbitrarily  high. 

3.  The  algorithm  is  complete  only  in  any  of  the  following  cases: 

•  The  matrices  representing  plausibility  of  nodes  never  leaves  that  plausibility 
undetermined.  That  is  for  every  state  of  the  parents  of  the  node  in  the 
network  there  is  a  unique  most  plausible  state. 

•  Nodes  whose  most  plausible  state  remain  uncommitted  are  not  involved  in 
cycles  where  all  other  nodes  are  also  uncommitted. 

In  the  context  of  the  algorithm,  completeness  means  that  if  the  probability  of 
A  =  a:  is  high,  then  the  algorithm  should  return  x  as  the  most  plausible  state  of  A . 
The  lack  of  completeness  simply  means  that  some  times  the  algorithm  will  remain 
uncommitted  regarding  to  the  most  plausible  state  of  a  variable  A ,  even  though 
this  state  could  be  determined  by  exact  (but  time  consuming)  computations.  We 
point  out  that  checking  whether  the  result  of  a  run  of  the  algorithm  for  a  specific 
query  is  complete,  is  0{E)  where  E  is  the  number  of  edges  in  the  network. 

This  algorithm  can  be  used  to  perform  prediction,  given  a  course  of  action,  in  a  domain 
represented  using  default  rules,  or  as  an  approximate  prediction  algorithm  in  proba¬ 
bilistic  representation.  The  approximation  is  done  as  proposed  in  [15],  by  transforming 
exact  probabilistic  statements  into  order-of-magnitude  probabilistic  statements.  Details 
about  the  algorithm  can  be  found  in  [22], 

This  algorithm  is  implemented  on  top  of  C-NETS  [11]. 

5.2.5  Bounded  Conditioning 

e-bounded  conditioning  is  a  method  for  the  approximate  updating  of  causal  networks. 
It  is  a  special  case  of  bounded  conditioning  [26],  which  ranks  the  conditioning  cases  in 
cutset  conditioning  according  to  their  probabilities  and  then  considers  the  more  likely 
cases  first.  The  contribution  of  e-bounded  conditioning  is  in; 

®The  uncertainty  in  the  domain  is  stored  in  matrices  associated  with  the  nodes  in  a  network.  A 
table  look-up  operation  implies,  in  the  worst  case,  examining  all  the  entries  of  a  particular  matrix.  The 
size  of  these  matrices  is  nevertheless,  bounded. 
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1.  proposing  a  particular  method  for  ranking  the  conditioning  cases  and 

2.  providing  some  guarantees  about  the  approximate  degrees  of  belief  it  computes. 

To  simplify  the  discussion,  we  will  ignore  the  existence  of  evidence,  therefore  focusing 
only  on  the  computation  of  Pr[x)  where  is  a  variable  in  the  causal  network. 

Cutset  conditioning  computes  the  following  sum  [35], 

Pr{x)  =  ^  Pr{x  A  s), 

S 

where  s  ranges  over  all  possible  states  of  a  loop  cutset.  The  problem  with  cutset  con¬ 
ditioning,  however,  is  that  the  number  of  cutset  states  is  exponential  in  the  size  of  the 
cutset.  To  deal  with  this  difficulty,  bounded  conditioning  [26]  orders  the  elements  of  the 
above  sum  according  to  their  values  so  that  elements  of  a  bigger  value  are  considered 
first. 

e-bounded  conditioning  splits  members  of  the  above  sum  into  two  classes,  those 
exceeding  a  particular  value  c  and  those  having  a  lesser  or  equal  value  to  e.  It  then 
ignores  the  latter  elements  of  the  sum. 

e-bounded  conditioning  applies  to  both  probabilistic  and  kappa  causal  networks,  but 
we  restrict  the  discussion  here  to  probabilistic  networks. 

The  Probabilistic  Case 

For  some  c,  e-bounded  conditioning  splits  the  conditioning  sum  as  follows: 

Pr{x)  =  ^  Pr{x  A  s)  -f  ^  Pr{x  A  s). 

3,Pr(s)>e  s,Pt{s)<c 

The  algorithm  then  ignores  'Es,Pr(s)<c  A  s),  thus  computing: 

Pr'{x)  =  ^  Pr{x  A  s). 

s,Pr(s)>c 

An  important  property  of  e-bounded  conditioning  is  that  we  can  judge  the  quality  of 
its  approximations  without  knowing  what  the  chosen  value  of  e  because  it  provides  an 
upper  and  a  lower  bound  on  the  exact  probability. 

To  completely  specify  e-bounded  conditioning,  we  need  to  show  how  to  identify  cutset 
states  that  have  probability  no  more  than  e,  which  is  discussed  next. 

Suppose  that  we  have  an  algorithm  such  that  for  each  variable  X  and  value  x  it 
will  tell  us  whether  Pr{x)  <  e.  Using  such  an  algorithm,  call  it  e-algorithm,  we  can 
identify  all  cutset  states  that  have  probabilities  no  more  than  an  e.  One  such  an  e- 
algorithm  for  prediction  is  the  one  we  discussed  earlier  for  order-of-magnitude  networks. 
This  algorithm,  which  is  linear  in  the  number  of  arcs,  is  sound  but  incomplete,  where 
incompleteness  means  that  the  algorithm  may  return  “unknown”  for  whether  Pr[x)  <  e. 
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This  incompleteness,  however,  does  not  matter  for  e-bounded  conditioning:  it  only  means 
that  some  cutset  states  which  have  probability  greater  than  c  are  being  considered  by  e- 
bounded  conditioning  although  they  should  be  ignored.  Of  course,  however,  the  closer  to 
being  complete  the  e-algorithm  is,  the  better  the  performance  of  e-bounded  conditioning 
will  be. 

6  Concluding  Remarks 

Military  planning  activities,  like  most  complex  problems,  must  respond  to  multiple  com¬ 
peting  objectives  with  uncertainty  about  actual  outcomes.  The  approach  pursued  at  the 
Rockwell  Science  Center  is  centered  on  well  founded  normative  notions  of  rational  deci¬ 
sion  making  and  probability  theory,  and  is  based  on  the  expertise  of  the  researchers  of 
the  laboratory  in  this  area. 

The  results  of  this  program,  as  summarized  in  Tables  1  and  2,  have  produced  im¬ 
portant  advances  in  the  state  of  the  art  in  the  generation,  evaluation,  and  simulation  ol 
plans,  with  recognized  impact  in  both  the  scientific  and  application  oriented  community 
as  well  as  in  ARPA.  Moreover,  we  are  currently  investigating  the  application  (and  transi¬ 
tion)  of  the  methods  and  technology  developed  under  this  program  to  Rockwell  business 
divisions  in  the  areas  of  automatic  diagnosis,  optimal  repair,  and  resource  allocation. 
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ARNOLD  AF8  TN  37339 


COMMANOEP/USAISC 

attn;  asop-oo-tl 

8LDG  61301 

FT  HUACHUCA  AZ  85613-5000 


AIR  WEATHER  SERVICE  TECHNICAL  LIB 
FL  4414 

SCOTT  AF3  IL  62225-545S 


AFIWC/MSO 

102  HALL  BLVO  STE  315 
SAN  ANTONIO  TX  78243-7016 


SOFTWARE  ENGINEERING  INST 
TECHNICAL  LIBRARY 
5000  FORBES  AVE 
PITTS81IRGH  PA  15213 


DIRECTOR  NSA/CSS 
Ml  57 

9800  SAVAGE  ROAD 

FORT  MEADE  MO  21055-6000 
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NSA 

ATTN:  0.  ALLEY 
DIV  X911 

9800  SAVAGE  ROAO 
FT  MEADE  MD  20755-6000 

DOO 

R31 

9800  SAVAGE  ROAD 

FT.  MEAOE  MO  20755-6000 


DIRNSA 

R599 

9300  SAVAGE  RQAO 
FT  MEADt  HO  20775 


FSC/IC 

50  GRIFFISS  STREET 
HAHSCOM  AFS  MA  01731-1619 


ESC/AV 

20  SCHILLING  CIRCLE 
HANSCOM  APS  MA  01731-2816 


PL  2807/R£SeARCH  LIBRARY 
OL  AA/SULL 

HANSCOH  AF8  HA  01731-5090 


TECHNICAL  REPORTS  CENTER 
HAIL  OROP  0130 
BURLINGTON  ROAD 
BEDFORD  HA  01731 


DE'^ENSr  TECHNOLOGY  SEC  ADMIN  CDTSA) 
ATTN:  STTO/PATRICK  SULLIVAN 

400  ARMY  NAVY  DRIVE 
SUIT5  300 

ARLINGTON  VA  22202 

HS.  KAREN  ALGUTRE 

RL/C3CA 

525  BROOKS  RO 

GRIFFISS  AFB  NY,  13441-4505 


JAMES  ALLEN 

COMPUTER  SCIENCE  DEPT/BLDG  RM  732 
UNIV  0?=  ROCHESTER 
WILSON  atVD 
ROCHESTER  NT  14627 

MS  TIFFANY  WALKER 
DIGITAL  SYSTEMS  RSCH  INC 
4301  NORTH  FAIRFAX  DRIVE 
SUITE  725 

ARLINGTON  VA  22203 

YIGAL  ARENS 
USC-ISI 

4676  ADMIRALTY  WAY 
MARINA  OEL  RAY  CA  90292 


MR,  RAY  RARETSS 

THE  INST.  FOR  LEARNING  SCIENCES 
NQRTHWFST£!^N  UNIV 
1390  MAPLE  AVE 
EVANSTON  IL  60201 

MR,  JEFF  3ERLINER 
F3N  SYSTEMS  S  TECHNOLOGIES 
10  MOULTON  street 
CAMSRIDGE'MA  02133 


MARIE  A.  8IENK0WSKI 
SRI  ir4TEPNATIONAL 
333  RAVENSWOOO  AVE/EK  337 
MENLO  PRK  CA  94025 


OR  MARK  S.  aODDY 

HONEYWFLL  SYSTEMS  S  RSCH  CENTER 
3660  TECHNOLOGY  DRIVE 
MINNEAPOLIS  MN  55413 


PIERO  30NISS0NE 

SE  CORPORATE  RESEARCH  £  DEVELOPMENT 
BLDG  Kt-RM  5C-32A 
P.  0.  SOX  8 
SCHENECTADY  NY  12301 

MR.  DAVID  BROWN 
MITRE 

EAGLE  CENTER  3,  SUITE  3 
0*FALL0N  IL  62269 


m.  MARK  SURSTEIN 
8BN  SYSTEMS  f.  TECHNOLOGIES 
10  MOULTON  STREET 
CAMBRIDGE  MA  02133 


MR.  GREGG  COLLINS 
INST  FDR  LEARNING  SCIENCES 
1890  MAPLE  AVE 
EVANSTON  IL  60201 


MR.  RANDALL  J.  CALISTRI-YEH 
GRA  CORPORATION 
301  DATES  DRIVE 
ITHACA  NY  14850-1313 


DR  STEPHEN  E.  CROSS 
SCHOOL  OF  COMPUTER  SCIENCE 
CARNEGIE  H^^LLON  UNIVERSITY 
PITTS3URGH  PA  15213 


MS.  JUDITH  DALY 
ARPA/ASTO 

3701  N.  FAIRFAX  DR.,  7TH  FLOOR 
ARLINGTON  VA  22209-1714 


THOMAS  CHEATHAM 
HARVARD  UNIVERSITY 
DIV  OF  APPLIFD  SCIENCE 
AIKEN,  RM  104 
CAM3RIOGF  MA  02133 

MS.  LAURA  DAVIS 
CODE  .5510 

NAVY  CTR  FOR  APPLIED  RES  IN  AX 
NAVAL  RESEARCH  LABORATORY 
NASH  DC  20375-5337 

MS.  GLADYS  CHOW 
COMPUTER  SCIENCe  DEPT- 
UNTV  OF  CALIFORNIA 
LOS  ANGELES  CA  90024 


THOMAS  L.  DEAN 
9ROWN  UNIVERSITY 
0£PT  OF  COMPUTER  SCIENCE 
P.O.  30.x  1910 
PROVIDENCE  RT  0291.2 
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LESLEY  CHU 

COMPUTER  SCIENCE  DEPT 
UNIV  OF  CALIFORNIA 
LOS  ANGELES  CA  90024 


MR.  ROBERTO  DESIMONE 
SRI  INTERNATIONAL  CEK335) 
333  RAVENSWOOD  AV £ 

MENLO  PRK  CA  94025 


PAUL  R.  COHEN 

UNIV  Q*=  MASSACHUSETTS 

COINS  OEPT 

LEDERLE  GRC 

AMHERST  MA  01003 


MS,  MARIE  DEJAROINS 
SRI  INTERNATIONAL 
333  RAVENSWOOa  AVENUE 
MENLO  PRK  CA  94025 


JON  DOYLE 

LABORATORY  FOR  COMPUTER  SCIENCE 
MASS  INSTITUTE  OF  TECHNOLOGY 
545  TtCriNOLOGY  SQUARE 
CAMSRIOGF  MA  02139 

OR.  BRIAN  ORABSLE 
AI  APPLI-CATIONS  INSTITUTE 
UNIV  OF  FDINeURGH/80  S.  BRIDGE 
FDINSURGH  EHl  LHN 
UNITED  KINGDOM 

MR,  SCOTT  FOUSE 
TSX  CORPORATION 
4353  PARK  TERRACE  DRIVE 
WESTLAKE  VILLAGE  CA  91361 


MR.  STU  DRAPER 
MITRE 

EAGLE  CENTER  3,  SUITE  8 
0*FALL0N  IL  62269 


MARK  FOX 

OEPT  0  INDUSTRIAL  ENGRG 
UNIV  OF  TORONTO 

4  tadole  creak  road 

TORONTO,  ONTARIO,  CANADA 


HR.  GARY  EDWARDS 

4353  PARK  TERRACP  DRIVE 

WESTLAKE  VILLACA  91361 


MS.  MARTHA  FARIWACCI 
MITRE 

7525  CQLSHIRE  DRIVE 
MCLEAN  VA  22101 


MR.  RUSS  FREW 
SENEGAL  ELECTRIC 
MOORE5TQWN  CORPORATE  CENTER 
3LDG  AIR  145-2 
MOOREST.QWN  NJ  08057 

MICHAEL  FEHLING 
STANFORD  UNIVERSITY 
ENGINEERING  ECO  SYSTEMS 
STANFORD  CA  •'>4305 


MR.  RICH  FRITZSQN 

CENTER  OR  AOVANCEO  INFO  TECHNOLOGY 
UNISYS 

P.O.  BOX  517 
PADLT  PA  19301 

MR  KRISTIAN  J.  HAMMOND 
UNIV  OF  CHICAGO 
COMPUTER  SCIENCE  0FPT/RY155 
1100  E.  58TH  STREET 
CHICAGO  TL  60537 

MR.  ROSEPT  F5?aST 
MITRE  COPP 

WASHINGTON  C3  CENTER,  MS  644 
7525  COLSHIER  ROAO 
MCLEAN  VA  22101-34S1 

RICK  HAYFS-ROTH 
CIHFLEX-TEKNOWLEDGE 
1810  EMSARCAOERO  RO 
PALQ  ALTO  CA  94303 


RANOY  GARRETT 

INST  FOR  DEFENSE  ANALYSES  (IDA) 
1801  N.  BEAUREGARD  STREET 
ALEXANDRA  VA  22311-1772 
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m.  JIM  HENOLER 
UNIV  OF  MARYLAMD 
DERT  OF  COMPUTER  SCIENCE 
COLLEGE  PARK  MO  20742 


MS.  YOLANDA  GIL 
U5C/ISI 

4676  ADMIRALTY  WAY 
MARINA  DEL  RAY  CA  90292 


MR. MAX  HFRION 

ROCKWELL  international  SCIEMCS  CTR 
444  HIGH  STREET 
OALO  ALTO  CA  94301 


HR.  STEVE  GOYA 
OISA/JIEO/GSll 
CODE  TOO 

11440  ISAAC  NcWTON  SO 
RESIGN  VA  22090 

HR.  MORTON  A.  HIRSCHB£RG»  OIPECTOR 
US  A^MY  RESEARCH  LASORATORY 
ATTN;  AMSRL-CI-C3 
ABERDEEN  PROVING  GROUND  HD 
21005-6066 

MR  -  MARK  A .  HQFFM  AM 
ISX  CORPORATION 
1165  NORTHCHA5F  PARKWAY 
MARIETTA  GA  30067 


HR,  RON  LARSEN 

NAVAL  CMG»  CONTROL  S  OCEAN  SUR  CTR 
RESEARCH,  DEVELOP,  TEST  S  '"rVAL  OIV 
CODE  444 

SAN  DIEGO  CA  92152-5000 

DR.  JAMES  JUST 
MITRE 

DEPT.  W032-M/S  Z360 
7525  COLSHIEP  RO 
MCLEAN  VA  22101 

MR.  CRAIG  KNO BLOCK 
use- 1  SI 

4676  ADMIRALTY  WAY 
marina  del  ray  CA  90292 
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^R.  RICHSRn  LOWE  <AP-iO) 
SRA  CORPORATION 
2000  ICTH  STPSET  NORTH 
ARLINbTOH  VA  22.201 


MR.  TED  C.  KRAL 
03N  SYSTEMS  0  TECHNOLOGIES 
4015  HANCOCK  STREET,  SUITEE  101 
SAN  niSGO  CA  92110 


MR,  JOHN  LOWRENCE 

SRI  INTERNATIONAL 

ARTIFICIAL  INTELLIGENCE  CENTER 

333  RAVENSWOOO  AVE 

MENLO  ^AS'K  CA  94025 

OR.  ALAN  MEYROWITZ 

NAVAL  RESEARCH  LABOR ATORY/COOS  5510 
4555  overlook  AVE 
WASH  DC  20,375 


ALICE  MULV5HILL 
MITRE  CORPORATION 
BURLINGTON  RD 
M/S  K-302 
BEOFORO  01730 

ROBERT  MACGREGOR 
USC/ISI 

4676  AOHIRAL'^y  WAY 
MARINA  OFL  REY  CA  90292 


WILLIA.M  S-  HARK,  MGR  AI  CENTER 
LOCKHEED  MISSILES  S,  SPACE  CENTER 
1801  PAGE  MILL  80 
PALO  ALTO  CA  94304-1211 


RICHARO  martin 

SOTWARE  ENGINEERING  INSTITUTE 
CARNEGIE  MELLON  UNIV 
PITTSBURGH  16213 


DREW  MCDERMOTT 
TALE  COMPUTER  SCIENCE  DfOT 
P.O.  80X  2158,  YALE  STATION 
51  PRO^’SPECT  STREET 
MEW  HAVEN  CT  06520 
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HS.  CECILE  PARIS 
USC/ISI 

4676  ADMIRALTY  WAY 
MARIMA  OTL  RAY  CA  D0292 


DOUGLAS  SMITH 
KcSTREL  I MSI I TUTS 
3260  HTLLVieW  AVS 
PALO  ALTO  CA  94304 


DR.  AUSTIN  TATE 

AI  APPLICATIONS  INSTITUTE 

UNIV  OE  EDTNPyPGH 

80  south  SRIOGE 

FOINBUPGH  EHl  IHN  -  SCOTLAND 

EOWARD  THOMPSON 
ARPA/SISTQ 

3701  N.  FAIRFAX  DR.,  7TH  FL 
ARLIIJGTON  VA  22209-1714 


NR.  STrPHEN  F.  SMITH 
ROBOTICS  INSTITUTE/CMU 
SCHENLFY  PPK 
PITTSSIJRSH  PA  15213 


DR.  A3RAHAM  NAKSMAN 
AFOSR/NM 

110  DUNCAN  AV£.,  SUITE  SHE 
BULLING  AF5  DC  20331-0001 


JONATHAN  P. STILLMAN 
GENERAL  ELECTRIC  CRO 
1  RIVER  PO,  RM  K1-5C31A 
P.  0.  BOX  8 
SCHENECTAOr  NY  12345 

MR,  ECWARO  C,  T.  WALKER 
SEN  SYSTEMS  t  TECHNOLOGIES 
10  MOULTHN  STREET 
CAM3RIDGE  MA  02133 


MR,  BILL  SWARTQUT 
USC/ISI 

4676  ADMIRALTY  WAY 
MARINA  0=L  RAY  CA  90292 


1 


GIO  WIFOFRHOLO 
STaNP0S?0  U^IIVERSITY 
OEPT  OF  COMPUTER  SCIENCE 

438  Margaret  jacks  hall 

STAMFORD  CA  94305-2140 

KATIA  SYCARA/THE  ROaOTICS  INST  1 

SCHOOL  OF  COMPUTER  SCIENCE 

CARNEGIE  HELLQN  UNTV 

DOHERTY  HALL  RH  3325 

FITTSaURGH  FA  15213 

MR.  DAVID  E.  WILKINS  1 

SRI  INTERNATIONAL 

ARTIFICIAL  INTELLIGENCE  CENTER 

333  RAVENSWQOD  AV£ 

MENLO  3ARK  CA  94025 

OR.  PATRICK  WINSTON  1 

MASS  INSTITUTE  OF  TeCHNOLQGY 

RM  NE43-317 

545  TECHNOLOGY  SQUARE 

CAMSRIDGF  MA  02139 

HUA  YANG  ^ 

COMPUTCS  SCIENCE  DEPT 
UNIV  OF  CALIORNIA 
LOS  ANGELES  CA  90024 


LTCOL  DAVE  NFYLAWO  I 

ARPA/ISTO 

3701  N,  FAIRFAX  DRIVE,  7TH  FLOOR 
ARLINGTON  VA  22209-1714 


MR.  RICK  SCHANT2  1 

83N  SYSTEMS  C  TECHNOLaGIES 
10  MOULTON  STREET 
CAMBRIDGE  MA  02133 


LTC  FRCO  M.  RAWCLIFFE  1 

USTRANSC0M/TCJ5-SC 

3LDG  1900 

SCOTT  AF8  XL  62225-7001 


JOHN  P.  SCHILL  t 

NAVAL  COMMAND,  CONTROL  &  OCEAN 

SURVEILLANCE  CENTtR/COOc  423 

EVALUATION  OIVISION 

SAN  DIEGO  CA  92152-5000 
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MR.  DOMfiLD  F.  RQ3SRTS 
RL/C3Ca 

525  5R30KS  ROAD 

GRIFFISS  A-a  MY  13441-4505 


ALLEM  SEARS 
HITRE 

7525  COLFSHIRE  DRIVE,  STOP  223*3 
MCLEAN  VA  22101 


STEVE  ROTH 

CEMTSR  FOR  INTEGPAT£n  MAMUFACTURI 
THE  R0*’3TICS  INSTITUTE 
CARNFbTE  MELLON  UMTV 
PITTSSURGH  PA  15213-33*30 

J£PF  ROTHcNBERG 

SENIOR  COMPUTER  SCIENTIST 

THE  RAND  CORPORATION 

1700  MAIM  STREET 

SANTA  MONICA  CA  90407-2138 


YOAV  SHOHAM 
STANFORD  UNIVERSITY 
COMPUTER  SCITNCc  DEPT 
STANFORD  CA  94305 


MR.  DAVID  0.  SK.ALAK 
UNIV  OF  MASSACHUSETTS 
OEPT  OF  CQMPUT-R  SCIENCE 
RM  243,  L3RC 
AMHERST  MA  01003 

MR.  s«IKt  ROUSE 
ArSC 

7800  HAMPTON  RD 
NORFOLK  VA  23511-6097 


MR.  DAVID  E.  SMITH 
POCKWELL  INTERNATIONAL 
444  HIGH  STREET 
PALO  ALTO  CA  94,301 


J£FF  ROTH£N3=RG 

SENIOR  COMPUTcP  SCIENTIST 

THE  PAMD  CORPORATION 

1700  MIN  STREET 

SANTA  MONICA  CA  90407-2133 


OR  LARRY  BTRNBAUM 
NGRTHs^FSTERN  UNIVERSITY 

Its 

1890  MAPLE  AV£ 

PVANSTON  XL  6Q201 

MR  RANDALL  J.  CALISTRI-YEH 
ORA 

301  OATES  DR 
ITHACA  NY  14850-1313 


MR  WESLEY  CHU 
CUS^PUTER  SCIENCE  DEPT 
UNIVERSITY  CALI"QPHIA 

LQS  ANGELES  CA  9002 


MR  PAUL  R  COHEN 
UNIVERSITY  OF  MASSACHUSETTS 
COINS  OE'^T,  LEDcRLE  GRC 
AMHERST  MA  01003 


MR  DON  EDDINGTON 

NAVAL  CQ!^MANO,  CONTROL  &  OCEAN 

SURV  CrNTEP 

POTL"  DIVISION,  CODE  404 
SAM  DIEGO  CA  92152-5000 

MR.  LE=  =RMAN 
CIMFLEX  TECKNQWLFOGE 
1310  SM3ARCARDERO  RO 
«’ALO  ALTO  CA  94  30  3 


MR  DICK  fstraDA 
OaN  SYST-MS  S  TtCHNGLDGIFS 
10  MOULTON  ST 
CAMBRIDGE  ma  02133 


HR  HARRY  PHRSOTCK 
BBM  SYSr~MS  AND  TECHNOLOGIES 
10  MDULTON  ST 
CAMS^’IDGE  MA  02133 


MR  MATTHEW  L.  GINS3SR& 
CIRL,  1269 

UNIVERSITY  QF  OREGON 
EUGENE  OR  97403 
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MR  IRA  GQLDSTSI^j 

OPEN  SW  FOUNDATIQM  RESEARCH  INST 
ONE  CAMBRIDGE  CENTER 
CAMBRIDGE  MA  02142 


MR  MOI5ES  GaLDSZMIDT 
INFORMATION  AND  OSCISION  SCIENCES 
ROCKWELL  INTL  SCIENCE  CENTER 
444  HIGH  ST,  SUITE  400 
PALO  ALTD  CA  94301 

MR  JEF-  GROSSMAN,  CO 
NCCOSC  RDT5  OIV  44 
5370  SILVERGATE  AVE,  ROOM  1405 
SAN  DI=GO  CA  92152-5146 


JAN  GUNTHER 

ASCENT  TECHNOLOGY,  INC. 
64  SIDNEY  ST,  SUITE  380 
CAM3RI0GF  MA  02139 


OR  LYNETTS  HTRSCHMAN 
MITRE  CORPORATION 
202  BURLINGTON  RO 
BEDFORO  MA  01730 


MS  A DELE  E.  HOWE 
COMPUTER  SCIENCE  OEPT 
COLORADO  STATE  UNIVERSITY 
FORT  CDLLINS  £0  80523 


OR  LESLIE  PACK  KAEL3LING 
COMPUTER  SCIcNCS  OEPT 
DROWN  UNIVERSITY 
PROVIDENCE  RI  C2912 


SU38ARA0  KAMBHAMPATI 
OEPT  0^  COMPUTER  SCIENCE 
ARIZONA  STATE  UNIVERSITY 
T£MP=  AZ  85237-5406 


MR  THOMAS  E.  KA2MIERCZAK 
SRA  CORPORATION 
331  SALEM  PLACE,  SUITE  200 
=AIRVIEW  HEIGHTS  IL  62208 


s’RADSEP  K,.  KHOSLA 
AR5>a/SSTO 

3701  n.  “AIR^^AX  OR 
ARLIMGTQN  VA  22203 


np.  CRAIG  KMO^LOCK 

use- 1ST 

4676  AOMIRALTY  WAY 
f^ARINA  DEL  RAY  CA  90292 


OR  CARLA  LUDLUW 
ROHE  LA30RATQRY/C3CA 
525  BROOKS  RD 

GRIF'ISS  A“8  NY  13441-4505 


OR  MARK  T.  ?3AY^URY 
ASSOCiaT?^  DIRECTOR  0*==  41  CCNTEP 
A0VAMC50  lOFO  SYSTcMS  TECH  G041 
MITR=  CGRP*  BURLINGTOM  RO ,  MS  K-3 
BEOFORO  MA  01730 

m  DONALO  P.  MCK4Y 
PARAMAK/UNISYS 
p  0  BOY  517 
PAOLI  PA  19301 


OR  KARON  MYEPS 
AI  CENTER 
SRI  INTEPNTIONAL 
333  PAVEMSyOOD 
MENLO  PARK  CA  94025 

OR  MARTHA  E  pQLLACK 
DEPT  G-  COMPUTER  SCIENCE 
university  op  PITTSBURGH 
PITTS3URGH  PA  15250 


PAJ  REOOY 

SCHOOL  0-  COMPUTER  SCIENCE 
CARNEGIE  MELLON  UNIVERSITY 
PITTSBURGH  PA  15213 


EDWINA  RISSLAND 

DEPT  OP  COMPUTER  L  INFO  SCIENCE 
UNTVERSI'^Y  OF  MASSACHUSETTS 
AMHERST  MA  01003 
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m  NnRMA^4  SADEH 
CIMOS 

THE  PO^^QTICS  INSTITUTE 
CAPNEGIE  MELLON  UNIVERSITY 
PITTSBURGH  PA  15213 

MR  E^^IC  TIFFANY 
ASCENT  TECHNOLOGY  INC. 

237  LCNGYieW  TERRACE 
WILLIAMSTQWN  MA  01267 


MANU?=^La  VELQSa 
CARNEGIE  MELLON  UNIVERSITY 
SCHOOL  OF  COMPUTER  SCIENCE 
PITTSBURGH  PA  15213-33G1 


DAN  W^LD 

DEPT  0=  COMPUTER  SCIf'NCc  C  £N 
MAIL  STOP  FR-35 
UNIVERSITY  OF  WASHINGTON 
SEATTLE  WA  9P1F5 

MR  CRAIG  WI£P 
ARFA/SISTO 
3701  N.  t^ATRFAX  DR 
ARLINGTON  VA  22203 


MR  JOE  RD3ERT5 

ISY  CORPORATION 

2231  CRYSTAL  DRIVE,  SUITF  500 

ARLINGiaN  VA  22202 


COL  JG«N  A.  WARDEN  III 
ASC/CC 

225  CHENNAULT  CIRCLE 
MAXWELL  Ar?  AL  36112-6426 


DR  TOH  GARVEY 
ARFA/SISTQ 

3701  NPRTH  FAIRFAX  DRIVE 
ARLINGTON  VA  22203-1714 


MR  JOHN  N.  FMTZMINGER,  JR. 
ARPA/DIRO 

3701  north  FAIRFAX  DRIVE 
ARLINGTON  VA  22203-1714 


LT  COL  ANTHONtf  WAISANEN,  ?HO 
COMMANO  ANALYSIS  GROUP 
HQ  AIR  HOBILITY  COHHAHD 
402  SCOTT  HRIve,  UNIT  3L3 
SCOTT  AFO  IL  62225-530T 

DIRECTOR 

AR^’A/STSTO 

.3701  north  FAIRFAX  OPIVF 
ARLINGTON  VA  22203-1714 


MS  LFSLI^  WILLIAMS 
DIGITAL  SYSEMS  RSCH  INC 
4  301  NOR'^H  FAIRFAX  DRIVE 
SUITE  ^25 

ARLINGTON  VA  22203 

DECISIONS  S  DESIGNS  INC. 
-ATTN:  ANN  MARTIN 

8219  LF,ES3URG  PIKE 
SUIT?;  390 
VIENNA  VA  22132 


*U.S.  GOVERNMENT  PRINTING  OFFICE:  1995-610-126-50126 
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MISSION 

OF 

ROME  LABORA  TORY 


Mission.  The  mission  of  Rome  Laboratory  is  to  advance  the  science  and 
technologies  of  command,  control,  communications  and  intelligence  and  to 
transition  them  into  systems  to  meet  customer  needs.  To  achieve  this, 
Rome  Lab: 

a.  Conducts  vigorous  research,  development  and  test  programs  in  all 
applicable  technologies; 

b.  Transitions  technology  to  current  and  future  systems  to  improve 
operational  capability,  readiness,  and  supportability; 

c.  Provides  a  full  range  of  technical  support  to  Air  Force  Materiel 
Command  product  centers  and  other  Air  Force  organizations; 

d.  Promotes  transfer  of  technology  to  the  private  sector; 

e.  Maintains  leading  edge  technological  expertise  in  the  areas  of 
surveillance,  communications,  command  and  control,  intelligence,  reliability 
science,  electro-magnetic  technology,  photonics,  signal  processing,  and 
computational  science. 

The  thrust  areas  of  technical  competence  include:  Surveillance, 
Communications,  Command  and  Control,  Intelligence,  Signal  Processing, 
Computer  Science  and  Technology,  Electromagnetic  Technology, 
Photonics  and  Reliability  Sciences. 


